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ABSTRACT 

The issue of how to react quickly to the educational 
needs arising from technological change has been deemed a central 
problem facing Technical and Further Education (TAFE) in Australia. 
Therefore, a national study examined various curriculum research 
methods that hold promise for speeding up the curriculum research and 
development process. The first phase of the study consisted of a 
description and analysis of group process methods. The following 
methods were analyzed: the Nominal Group Technique, the Developing a 
Curriculum Process (DACUM), the critical incident technique, Force 
Field Analysis, and brainstorming. It was concluded that the Search 
Conference, Delphi and exploratory methods, and, to a lesser extent, 
the Nominal Group Technique hold promise for providing data for 
adapting curriculum to occupational structural change. DACUM, while a 
promising method of obtaining data in the cognitive domain, may need 
to be linked to other methods that are more sensitive to change when 
used as a curriculum research procedure on occupations undergoing 
structural change. The critical incident technique can be 
advantageous in curriculum research for short courses, and Force 
Field Analysis appears best suited for use as an adjunct to other 
group process methods. In addition to describing the research project 
and its findings, this report includes several chapters of practical 
guidelines on using and blending the various group process curriculum 
research methods. Appendixes include descriptions of some 
questionnaire methods used and a sample search conference, 
observational studies, and a DACUM chart. Four pages of references 
and figures and tables are included. (MN) 
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FOREWORD 



A central problem facing TAFE in the 1980s is how to react quickly to the 
educational needs which arise from technological change? 

This national study was designed to investigate TAFE curriculum research 
methods which hold promise for speeding up the curriculum research and 
development process. The study was planned to have three linked components: 
(1) description and analysis of group process methods, noting suitability, 
strengths, weaknesses, requirements for implementation, pitfalls and so on; (2) 
evaluation of methods by comparison of carefully monitored case studies on 
the same occupational grouping in separate States/Territories; and (3) 
development of methods, in effect a fine-tuning of the methods suitable for 
TAFE usage. 

This report represents the completion of the first component. For various 
reasons beyond the control of the researchers, the case studies necessary for 
completion of components (2) and (3) have not yet been pursued. 

Where possible the authors have attempted to describe each research method 
in sufficient detail for an educator or researcher to apply them, though some 
methods are described in greater detail than others. 

The authors undertook the study whilst working in the Curriculum Services 
Directorate of the NSW Department of TAFE; Neil Jones as Senior Education 
Officer and Dr Tony Anderson as Education Officer. The authors would be 
interested to hear of any studies, particularly of an evaluative nature, based on 
the methods outlined in this report. 

This report includes, in Part V, a summary of the findings of a companion 
study by the same authors examining the role of the facilitator in planning and 
conducting curriculum research group process methods, titled: The Facilitation 
of Curriculum Research Workshops in TAFE, also published by the TAFE 
National Centre for Research and Development. 



ABSTRACT 



This TAFE national project examines ways in which technica 1 and further 
education curriculum research can be speeded-up in the data collection and 
analysis phases. After discussing the need for preliminary investigation the 
study describes and analyses the following research procedures: exploratory 
(qualitative) research, the Search Conference Method, the DACUM (developing a 
curriculum) Method and derivatives of DACUM, the Nominal Group Technique, 
the Delphi Method, the Critical Incident Technique and Force Field Analysis. 
The methods were chosen after reviewing the literature and research methods 
used in TAFE. Since these methods provide different perspectives on 
occupational analysis and therefore yield different types of data, some ways in 
which they can be used in combination are covered. A guide to the selection 
of the methods is provided. 

The planning and implementation of curriculum group-process methods is 
summarised in terms of the value orientations (overarching principles which 
guide choices between alternatives) and social rules (practical guides for action) 
which a group facilitator may adopt. A model of the general stages and steps 
in planning and conducting curriculum research workshops is included. Some 
issues which arise in the use of group-process methods, such as strategies for 
'conflict resolution 1 , are highlighted. 

Appendices cover some questionnaire methods for occupational analysis, a 
program for a short Search Conference, a brief description of an observational 
method for the study of skill masters, and some examples of DACUM charts. 

The methods reviewed offer, in varying degrees, ways of shortening the 
curriculum research and data analysis process. The Search Conference, Delphi 
and exploratory methods and, to a lesser extent, the Nominal Group Technique 
(NGT) hold promise for providing data to enable curriculum to be adapted to 
occupational structural change. 

The NGT method appears to offer a useful approach for identifying the 
issues to be addressed in curriculum development and is a useful 'conflict 
resolution' procedure in certain situations. 

The DACUM method, regarded as a quick and effective means for analysing 
occupational education needs at the psycho-motor level, holds promise, when 
linked to Delphi-type processes, of providing data in the cognitive domain (e.g. 
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central educational concepts and the knowledge/skills needed to carry these 
concepts into effective task performance on the job). DACUM is also 
promising for detailing 'the -ihef e and now' of work functions but, in its 
traditional form, appears less effective, compared with the Search Conference, 
Delphi and exploratory research, in detecting change within occupations. 
DACUM, therefore, may need to be linked to other methods more sensitive to 
change when used as a curriculum research procedure on occupations 
undergoing structural change. Or it may need to include an additional segment 
which looks at trends within the occupation, possibly at the end of the DACUM 
session. 

The Critical Incident Technique is seen as offering advantages in curriculum 
research for short courses, especially where urgent information is sought on the 
immediate, practical knowledge needed by students. It can also be used in 
assessing what 'critical' knowledge needs to be included in a course for 
assessing students' perceptions of the relevance of a program to their needs. 

Force Field Analysis, since it enables a planning group to highlight the 
strengths and weaknesses of the forces affecting the implementation of change, 
appears best suited for use as an adjunct to other group process methods or in 
teacher workshops examining how occupational data can be translated into a 
curriculum. 
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ORIENTATION 



1.1 BACKGROUND TO THE RESEARCH PROJECT 

At the commencement of this project in 1983 a number of TAFE researchers 
and curriculum developers had begun to show interest in and to use research 
methods which are relatively new to TAFE, including methods currently in use 
by sociological and educational researchers outside TAFE. Some of these 
methods depend less upon the collection of masses of detailed data about 
occupational work functions than has traditionally been the case in TAFE, and 
focus more on occupational structure and change. Moreover, there has been 
substantial pressure from outside TAFE for TAFE to adopt faster and less 
costly methods for ascertaining the educational needs of an occupation. 
Indeed, industry representatives on curriculum advisory committees and 
Apprenticeship Authorities and Labour and Industrial Commissions, have for 
some years been openly critical of the time and costs involved in many of 
TAFEs responses to problems requiring the analysis of occupational or 
community needs for curriculum development purposes. 

In view of these contributing factors it was opportune for TAFE to 
undertake research which would focus on an analysis of research techniques 
which promised 'fast-response 1 and 'effectiveness 1 for the purposes of curriculum 
design and review. 

This report presents a review of a number of methods. Most use workshop 
or seminar approaches. Other methods have been included, such as exploratory 
research, the Critical Incident Technique and observational studies of skill 
masters, as these have particular strengths when used in combination with 
group process methods. We have not dealt with questionnaire methods, other 
than to briefly describe some in Appendix A, because we see these methods as 
options to be used following group process or other methods where validation is 
required of the course directions set by group process or exploratory methods. 
It should be stressed that some of the methods discussed here have not been 
trialled extensively in vocational curriculum research in TAFE. These include 
the Search Conference method, the Nominal Group Technique, and the Critical 



Incident Technique. There would be an opportunity, therefore, to continue the 
work begun in this project to address the remaining two components that were 
originally envisaged: 

1. Evaluation of the methods derived from comparative 
case studies including a consideration of the suitability 
of the methods to particular TAFE study areas, target 
populations, occupational areas or curriculum problems, 
as appropriate, 

2. Fine-tuning of the methods suitable for use in TAFE 
including a commentary on the professional development 
needs of potential users of the methods, 

1.2 SCOPE AND AIMS 

This study aims to inform curriculum developers of a range of occupational 
research methods for use in TAFE which purport to offer a shorter response 
time in data collection and analysis for curriculum development in technical 
and further education. The methods covered are: 

. Preliminary investigation 
. Exploratory (qualitative) research 
. The Search Conference Method 
. The Nominal Group Technique 
. The Delphi Method 

. The DACUM (developing a curriculum) method 
and derivatives 

. The Critical Incident Technique 

. Force Field Analysis 

Each method has been presented with the aim of providing intending users with 
sufficient information about the uses of the method, the data produced, how 
the method is used, and what are its pitfalls. Where possible, details of 
duration, cost, time and resources needed, have been included. Since the 
methods covered provide different types of data, ways in which the methods 
may be blended or used in combination, are discussed. 

Most of the methods described use group-process techniques, which entail a 



•specialist 1 group meeting in a seminar or workshop setting to analyse an 
occupation. Therefore the report also includes a summary of a study of the 
role of the facilitator/group leader in the conduct of curriculum research group 
process methods (Anderson and Jones, 1986). 

Decisions about the choice of methods must always consider the amount of 
supporting evidence which an accreditation body may require for allocation of 
resources for a particular program. With the exception of the Delphi method, 
the study does not cover questionnaire methods for deriving and analysing 
occupational data. This is not to deny the effectiveness with which 
questionnaire methods can provide comprehensive information about need and 
demand for education. Questionnaire methods have a vital role to play and 
are widely used in TAFE especially where the decision to allocate substantial 
resources must be based on demonstrated need and demand. However, concern 
has been expressed within and outside TAFE about large-scale questionnaire 
surveys. This concern centres around: 

(a) the time taken to mount and process large surveys 

(•blockbusters'); 

(b) doubts about the reliance which can be placed on 

surveys with response rates of around 40-50 percent 
which mean that possible biases within the 
non-response group remain unknown; 

(c) uncertainty about the ability of questionnaires to 

provide an accurate picture of complex work functions 
in an industry, a problem facing any curriculum 
research method; 

(d) the difficulties experienced in data interpretation. 

Problems with interpretation arise because questionnaire methods which purport 
to examine the detail of tasks performed, no matter how thoroughly, often do 
not provide the data analyst with guides as to how this data is to be 
translated into a curriculum. This remains a question of interpretation which 
is often the subject of dispute during the curriculum design stage due to 
subjective bias. As Connell et al. (1982:26) point out, the only thing that 
orthodox surveys do well is to collect a few simple facts about largish numbers 
of people, at the expense of understanding. 

The problem of achieving an understanding of the nature of skills in 



occupational wort tad hew to teach these •kills it rtndtrtd difficult because 
occupeiionel work functions involve complex sod overlapping conceptual 
knowledge Md practical skills. The importance of obtaining a broad 
undent ending of an occupation in order to bate decisions about course 
structure, aims and content areas on research rather than on Interpretive 
gutters, is t alien up in the section on exploratory (qualitative) research methods 
(Section 2.1). 

Reservations about large scale surveys have led decision makers In TAFE to 
ask; In what ways can TAFE curriculum researchers Improve data quality and 
speed up the data collection and analysis process of curriculum research? 
Therefore, this study was commissioned to examine so-called 'fast-response' 
methods for occupational data collection and analysis for purposes of course 
design and review in TAPE. 

1.3 QUESTIONS WHICH GUIDE SELECTION OF CURRICULUM RESEARCH 
METHODS 

Table I.I sets out a number of issues and questions which guide selection of 
the methods covered by this study. Since each method hat specific strengths 
and weaknesses a summary of the meln features of the methods Is given in 
Section 1.4 and 1.5 prior to the detailed descriptions of the methods In 
Chapters 2 and 3. 

Broadly, there are three sets of Issues to be considered in selecting a 
research strategy. These concern content, methodology and the problems 
associated with introducing change. Content Issues include: 

1. The "what' versus the 'how' of curriculum. 

2. The relative emphasis which should be pieced on Identifying and learning 
skills at oppos e d to understanding the underlying concepts. 

3. The relative emphasis which should be placed on Identifying the future 
directions in which an Industry is moving and emerging skill requirements, as 
opposed to analysing the "here end now' of work In en Industry to Identify the 
skills end knowledge currently needed. 



Methodological Issues Include: 



1. The relative strengths of qualitative and quantitative research methods. 

2. Sampling bias In the selection of key Informants and workshop 
participants and the accuracy/reliability of data so obtained. This Is equally an 
Issue for all methods* 

3. To what extent should educators as well as practitioners be Involved In 
group process research methods. 

4. Who controls the workshop output (the tension between researcher 
directed focus and participant directed focus). 

Problems of introducing change are complex but two Issues need to be 
highlighted: (A) the problem of a providing sound rationale for resource 
allocation in times of constraints on funding and (B) the problems of selecting 
a curriculum research and development process which will facilitate 
implementation of a curriculum. The latter issue is related to the so called 
'change agent 1 role of researchers and educators and opens up questions of 
what is an appropriate consultancy role relationship between researchers, 
curriculum developers and curriculum implemented. (See Appendix B for an 
example of a consultancy model used for a fast-response curriculum 
development project in NSW TAFE.) 

There are no easy answers to any of these Issues. Content issues are taken 
up again In Chapter 4 when we discuss ways of blending research methods and 
methodology issues are taken up in Chapter 6. Table 1.1 lists some key 
questions to be borne In mind when selecting research methods. 



Table 1.1 

Questions which guide selection of so^alled 'fast -response' curriculum research 
methods. 



Key Question 



Curriculum Research Method 



What is currently available that 
bears on curriculum development 
in the area? 

Do you need to describe the 
general field to be covered by 
a course and to identify the main 
elements in this field and their 
interrelationships, or to 
identify the major questions 
which curriculum research 
should address? 

Is the occupation undergoing 
substantial change? 



Is the occupation relatively 
stable? 

Is there group tension about 
the direction a course 
review should take? 



Is urgent information needed 
about the critical knowledge 
needed to perform specific work 
functions? 

Does the data derived from any 
of the methods require 
validation or does the derived 
program require validation? 



Preliminary 
investigation 



Exploratory research 
Student profiling 
Graduate student survey 



Exploratory research 
The Delphi Method 
The Search Conference 

DACUM (or derivatives) 



Search Conference 
Nominal Group Technique 
(Possibly Force Field 
Analysis) 



Critical Incident Technique 



Questionnaire 



1.4 PRELIMINARY INVESTIGATION 

While preliminary research is, by nature exploratory, a distinction will be 
made between preliminary work of the kind described here and the techniques 
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of exploratory (qualitative) research described in Section 2.1. 

All curriculum development projects in TAFE should include some form of 
preliminary investigation. Preliminary investigation is important because 
sufficient information may be obtained which simplifies the curriculum research 
project by showing where gaps exist in available knowledge. Moreover, suitable 
curriculum materials or even a syllabus may be turned up during the 
preliminary investigation phase, enabling research strategies such as pre-testing 
or other forms of validation to be used if confirmation of the relevance of the 
syllabus is needed. 

The scope of preliminary work varies, of course, according to time and 
other constraints and may touch on some or all of the following areas, 
depending on the size and importance of the project. 

(a) Investigation of the relevance to curriculum design and implementation of 
legal or regulatory requirements bearing on the occupation including awards, 
codes of safe practice. 

(b) Identification of the strengths and weaknesses in the existing educational 
program including: views of teachers; evaluation of outcomes for graduates of 
the program; and development of a profile of students which may include: 

. Age, sex and educational background (including 
literacy and numeracy levels); 

. Work experience; 

. Work performed by the employer (broad categories); 
. Work performed by the student: 

- specific processes performed 

- variety of work performed 

- extent of control, if any, over the nature 
of the work including its scope and depth 

- opportunity for making decisions about the tasks 
performed, i.e. scope for putting own ideas 
into practice 

- students' views about shifts/trends within 
the occupation 
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- students' views about their career paths (e.g. 
stay 'on-tools', move into management or 
supervision, start own business) 

- students' views about strengths and weaknesses 
in the education received, including problems 

being encountered with the order in which the course 
is traversed, addressing, if needed, questions of 
whether knowledge which students see as vital to be 
acquired early in the program are placed, in their 
view, too far into the course (see Critical Incident 
Technique) 

- students' reasons for abandoning training 

- examination of students use of other educational 
sources. 

(c) Collection of relevant data from: Australian Bureau of Statistics (ABS); 
Australian Standard Classification of Occupations (ASCO); Kompass (industry) 
directory; research reports, e.g. survey or review reports, workforce forecasts, 
occupational analyses, instruction/training manuals, curriculum documents, 
teacher reports, theses, records of advisory committee meetings. 

(d) Identification of other curriculum work in the same or related fields, 
through literature searches including the TAFE National Clearinghouse 
publications: Initiatives in TAFE and Projects in Progress. 

(e) Consideration of TAFE policy in relation to the area (e.g. equality of 
opportunity, occupational health and safety, etc). Sources of TAFE policy 
including policy and/or curriculum units within TAFE; policy statements by 
senior officers; CTEC Triennium reports (TAFEC); Committee of Enquiry 
reports from oii.er government departments. 

(f) Identification ano consultation with key informants with a broad grasp or 
detailed knowledge of the area. 

(g) Identification and consultation with decision-makers, client groups, 
audiences and stakeholders: 

. within TAFE, such as schools, colleges, regions, teachers, 
students, senior officers, curriculum specialists; 

. outside TAFE, such as employer and employee groups, 
other government departments, other educational 
providers, National Training Council, Industry 
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Training Committees (see section on exploratory 
research). 

(h) Identification of the key influences which are impinging on the 
occupational area (technological, economic, structural, social) especially changes 
which are likely to take effect during the next ten years (see sections on 
exploratory research, the Search Conference and the Delphi methods). 

(i) Clarification of the areas which the research needs to cover and 
formulation of the research questions that the project will need to address (see 
section on exploratory research). 

(j) Finalise (or make recommendations) about the project research design 
including data sources, data collection and analysis methods, critical project 
decision points, timetabling, staffing and budget. 

1.5 SUMMARY OF METHODS 

Exploratory (qualitative) research methods 

Uses 

To ascertain the broad thrust of a course offering in terms of scope, 
weights of emphasis to be given to segments of the program, depth of 
coverage within subjects and the more important aspects of subject 
sequencing. Properly handled, exploratory research is particularly useful when 
little information is available about an occupation and for obtaining an 
understanding of the structural complexities of an occupation as a prelude to 
identifying specific data needs for curriculum design. 

Data collection method 

The first step is to decide the research questions which are to be explored 
in interviews with key informants or in group discussion. Research questions 
are the broad questions which guide and focus the data collection and analysis 
phases. Examples are: 'In what ways is technological change altering the 
pattern of work skills in an occupation? 1 and 'What should be the central 
outcomes of the level of course being designed/reviewed? 1 The research 
questions should span the occupational/social context level (the wider forces 
which shape immediate knowledge needs or future needs which the educational 
program would anticipate) and the individual level, so as to allow student needs 
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to be related to an understanding of how work in the occupation is organised. 

These research questions are usually divided into finer questions such as 
those used as probes in interviews. Since the aim of exploratory research is to 
explore, new questions are added as the process unfolds. The aim is to 
achieve a broad understanding of the occupation in terms of patterns or 
common themes but not to be deceived by initial appearances: to look for 
cracks or flaws in the emerging data and to follow up the questions that these 
pose. 

Sample 

This depends on the scope of the project and the research questions being 
studied. The composition of the sample should be decided according to what 
the researchers need to know. For example, if the problem for curriculum 
design is the existence of several generations of technology (latest to the 
oldest) in use in an occupation, sampling cases must include instances of all 
generations of technology so that curriculum developers can know the breadth 
of education required. 

Advantages 

The strength of exploratory research lies in its focus on the broader picture 
and on relating the educational needs at the individual level, to an 
understanding of occupational structure and change. The method's sensitivity t 
detecting change and the educational implications of change is of great 
advantage in preventing the problem of preparing a course which is obsolete by 
the time it is implemented. 

Disadvantages 

The method requires flexibility in implementation and places a premium on 
the ability to synthesise (join data) and to extrapolate (look beyond data). 
Since the usual data collection is by interview, skill is required in conducting 
relatively open and free-ranging interviews and analysing responses for main 
themes whilst remaining alert for cracks or flaws in the emerging data which 
should be followed up as leads for further exploration. 
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Cost and duration 



This is difficult to quantify since exploratory research should sample enough 
'typical applications' of the work in the occupation, to provide an overall 
picture of how the occupation is organised (structured) and how it is changing. 
Deciding the minimum but adequate number of sample cases depends on the 
complexity of the occupation. Experience in TAFE applications on small, 
relatively homogeneous occupations suggests that the number of sample cases 
for focussed interview and observation of work functions can be set at a 
minimum of 4-8. Once the broad picture has been obtained, the time required 
for curriculum development depends on the level of knowledge held by the 
syllabus writers. 

The Delphi Technique 

Uses 

To obtain consensus from a group of 'experts' about the likely pattern of 
future events and to show where these experts disagree. 

Data collection method 

A series of questionnaires are sent to the 'expert 1 panel. Their responses 
are summarised and returned to the panel until patterns of agreement and 
disagreement become clear. Usually two or three rounds of questionnaire 
administration and analysis are sufficient. The method can also be used during 
a conference, e.g. where a questionnaire is written during the conference, 
administered, analysed and the pooled results returned to the group for rating 
or comment. 

Sample 

People who are in a position to comment on likely future trends. Thirty or 
more respondents. 

Advantages claimed 

The method accesses people without requiring them to meet in person, 
saving travel time and cost. It describes what key decision makers believe is 
technologically possible given particular constraints. It gives participants equal 
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opportunities to set down th^ir perceptions of future events. It eliminates 
those psychological barriers to communication that can affect face-to-face 
group interaction. Participants have time for thinking and reflection before 
giving final judgments. 

Disadvantages 

The method assumes that the achievement of consensus within the panel, 
renders the data more believable and that 'he anonymous responses are more 
likely to be more objective than those generated through face-to-face group 
processes. The method may under or over-estimate the likelihood of future 
developments within an industry because individual desires may become imposed 
on the forecasting process. Cross impact between specific forecasts can be 
difficult to assess. 

Applications 

The method is a promising means for assessing the directions which experts 
(or informed observers) believe an industry or social process (e.g. education) 
will take. Such directions could include what technological innovations from 
the range available are likely to be taken up, and in what time frame. Such 
information cannot, of course, be proven. However, the ability of Delphi to 
detect patterns of change likely in the short term (1 to 5 years) could reduce 
the problem of curriculum obsolescence. Delphi can also be used to scan the 
'here and now 1 and identify consensus on what are the matters of concern and 
the most promising solutions. 

Due to the dangers in forecasting, it would appear sound practice to join 
Delphi to a method which addresses the 'here and now 1 of work performed in 
an occupation. Because it accesses acknowledged experts, Delphi would work 
best in curriculum development for areas undergoing rapid change, when no one 
knows the shape of things to come. It would also be a useful component of a 
multiple method approach to rethinking a problem, e.g. the nature of trade 
training in the late 1980s, or examining approaches to distance education. 

The DACUM method 

Uses 

To systematically develop a training program incorporating all aspects of the 
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curriculum (syllabus, assessment and teaching resources) based upon the 
compilation of a detailed task analysis (identification of job competencies) from 
the view of 'experts' via a structured group process. 

Data collection method 

Data on job competencies (duties, tasks, task elements) are collected from a 
panel of selected 'experts' in the occupational area being studied. In a 
structured workshop session managed by a skillful facilitator, the panel of 
experts is asked to define the occupational area in increasingly finer detail. 
Job categories are identified, followed by the major duties, component tasks 
and finally the skills, knowledge and attitudes related to the component tasks. 

The data are provided by the experts and recorded by the facilitator on 
cards or butcher's paper until all job competencies have been analysed and are 
presented in a DACUM chart which becomes the basis for curriculum 
development. 

Sample 

The DACUM workshop traditionally operates with about 6-14 experts, i.e. 
people who are actually performing or supervising the job under study. 
Participants in the workshop should be representative of different skill strata 
within the occupational group. 

Advantages claimed 

The method appears to be quick and easy to manage. It is fairly structured 
and business like and seems to have appeal to industry experts. It focuses on 
what the practitioner needs to be able to do on the job. The method is 
systematic and embraces the training model used extensively by the US Armed 
Forces. As such it readily suits the established philosophies of many curriculum 
researchers in TAFE in Australia and provides ordered data which are conducive 
to full scale curriculum development activities where an entire instructional 
system is required. 

Disadvantages 

It has been argued that an instructional systems (such as DACUM) approach 
to training (objectives model) is too narrow and too inflexible to research 
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effectively the learning requirements of adult students, because it focuses on 
an analysis of the job, rather than the learning styles of students. DACUM 
falls into this category. 

Since DACUM draws data solely from industry experts, it has been argued 
that a DACUM derived curriculum may be poorly implemented by teachers. 
That is to say, because teachers and students are not a part of the analysis 
and design process, they are not as committed to the curriculum when it is 
being implemented. To try to overcome this weakness, some TAFE versions of 
DACUM include teachers in the workshop process. Another weakness is that 
DACUM does not identify how an occupation is changing, though it is possible 
to add a concluding phase examining 'future trends 1 . 

Applications 

The DACUM method appears to be particularly suitable to researching 
occupational needs in established vocational areas i.e. for reviewing or revising 
existing TAFE programs. This is the case because the DACUM process begins 
with the identification of job classifications and from there systematically 
analyses the tasks involved in performance in the job. 

It is further argued by some commentators that DACUM works best when 
applied to an occupational area which is comprised of a fairly homogeneous 
workforce. It would appear other more search-oriented methods could be more 
effective in exploring an occupational area or community need where no 
educational provision (course) already exists. 

The Search Conference Method 

Uses 

The main uses of this method to date have been organisational development, 
i.e. setting directions for improving organisational responsiveness to clients. 
In an educational context, the procedure, in essence, aims to assist people to 
make effective decisions now by clarifying what educational program they want 
to deliver in the future^ taking account of likely future trends, and current 
constraints. 
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Data collection method 



The Search Conference method is a flexible program containing a number of 
optional stages following the first step which is a 'FUTURE SCAN 1 of forces 
which are shaping the future environment. The 'FUTURE SCAN' which focuses 
on events, trends or forces which are perceived to be shaping the future, may 
include segments in which the conference considers the 'MOST DESIRABLE' and 
'MOST PROBABLE' futures. The central principle underlying the Search 
Conference method is that plans made in response to a turbulent environment 
grow out of 

(i) shared ideals or values about what are the important goals 
of planning for the future of the system in question 

(ii) consideration of the constraints which must be taken into 
account in realistic planning. 

Next, the workshop breaks into small groups which take the product of the 
'FUTURE SCAN' and the 'MOST DESIRABLE' and the 'MOST PROBABLE' 
future. Or, depending on the situation, the whole group may select 6-10 
changes they judge as important and spell out their likely effects taking into 
account 'CONSTRAINTS' , and what to do about them. In some cases, the 
conference may examine the effects of changes or trends occurring nsiHe the 
school or college ('INTERNAL SCAN') which are shaping its future. Another 
optional step, recommended in curriculum research applications, is the 
'HISTORICAL RECONSTRUCTION' - a rewinding of the clock to 'see where we 
have come from?', in order to see what is good and should be preserved. The 
next step, consideration of present 'STRENGTHS AND WEAKNESSES' is to take 
stock of 'what we are good at' and 'what we are poor at?' By this time most 
of the key issues and options will have been identified and three steps remain: 

(a) ISSUES AND PRIORITIES: Placing priorities on the issues 

identified during the conference. 

(b) ACTION PLANNING: Selection of issues and the development 

of action plans by small groups. 

(c) FINAL SESSION: Report-back sessions on the action plans 

which have been formulated and discussion of 
implementation procedures. 

It is usual for the action groups to continue their work after the conference 
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ends and these groups may meet again or report to *\ich other through 
coordinator. 



Sample 

The essential requirement is representation of major 'stakeholders' (interested 
parties). In curriculum research applications these would include 
representatives of the occupational grouping, teachers, students, curriculum 
developers and other key decision-makers in relation to curriculum 
implementation. Sample size around 30 is ideal but Search Conferences can be 
run with as many as 150 people. 

Advantages claimed 

The method provides 'future-oriented' data from which to design a 
curriculut -.. This is particularly important in design of vocational curricula for 
occupational areas undergoing change. The process tends to 'open the horizons' 
of participants ('FUTURE SCAN' leading to 'DESIRABLE FUTURES'), while 
keeping a firm footing in relation to implementation realities ('CONSTRAINTS'). 
Because of the representation of 'stakeholders' in the conference and their 
involvement in the research and action planning stages, the method is said to 
offer enhanced prospects of implementation of outcomes. 

Disadvantages 

As yet we do not know how effective is the method in analysing the detail 
of competencies required to perform work in an occupational grouping. 

To effectively apply the method the facilitator requires a good understanding 
of how to work with a group in an exploration of the wider contextual forces 
impinging upon an occupational grouping. Moreover, each Search Conference 
must be tailored to the particular situation. This requires a detailed 
understanding of the method. 

Applications 

The ideal application is on an occupational grouping undergoing change. The 
method appears particularly suited to answering the broad questions in 
curriculum design such as course philosophy/rationale, aims, scope and course 
structure and can, it is said, be applied to trade and technician courses, as 
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well as short courses and organisational and personal development projects* 
The method has utility in circumstances where there are diverse views about 
what changes, if any, should take place in a curriculum* 

Nominal Group Technique (NGT) 

Uses 

. To define a problem. 

. To generate solutions* 

. To establish priorities for action* 

Data collection method 

The Nominal Group Technique (NGT) requires a small group who meet in a 
'brainstorming' session* The process begins with individuals silently writing 
ideas. These are then listed, one from each member in turn, on large sheets 
of pap.;. The ideas are 'collapsed' to remove overlaps, if needed* The list of 
items is ranked to establish group consensus as to priorities* Any additions are 
made, then the list is subjected to a final vote. The process is structured to 
limit group discussion in the idea-generating phase (silent writing of ideas) and 
the listing of these ideas (by limiting discussion to clarification only). 

Sample 

NGT appears to work best with groups of 8-10 people (several groups can be 
run concurrently). Participants can be those in a position to possess expert or 
up-to-date knowledge of the area and/or those who will be required to 
implement the (group) decision. 

Advantages claimed 

The method appears to be especially effective when the participants are 
strongly motivated to attempt the task (define the problem, or solve a known 
problem) and can tolerate the rules of the method, e.g. the rule of 'no 
discussion except for clarification.' The method claims to encourage equal 
participation and reduce the effects of one or two participants dominating the 
session. Moreover, because the method uses different procedures in the 
idea-generation and idea evaluation phases, it is claimed that the method 
improves both the speed of generation of ideas and the quality of their 




evaluation. The method alio gives • numerical weighting to the output and so 
can show what facets of tha problam ara taan at central, or what solutions 
aaam most promising* 

PsaaaVasHajai 

A lay to tha mat hod Is tha framing of tha NCT question which focuses the 
group's efforts. The ' OT question can be specific, e.g. Whet subjects in this 
trade course should be reviewed?* or broad, e.g. Whet will the plastics 
worker of the IMOs be required to know?' Rut, unless the NCT question Is 
appropriate, the process can be Impaired. Some experience Is needed, It 
appears, In formulating an NCT question which Is effective, centre! to the 
issue being addressed and one that participants can relate to and understand. 
The level of abstraction (generality) Involved In answering an NOT question 
may need to be specified. The method mey be less suitable for occasions 
when sharing and evaluating information, In open discussion, are Important in 
both generating solutions and setting the groundwork for smooth 
Implementation. The method can generate frustration If applied In an 
insensitive, excessively rule-guided way to e group who really need to talk out 
a problem ea a prelude to tackling it In a systemetlc manner. 

Force) field A— lysis 

Uses 



To generate strategies for Implementing change situations by Identifying the 
forces, both negative end positive thet ere retarding change. 



For each problem area or concern, the forces which might help In solving 
the problem are listed opposite the forces which might hinder the 
problem-eolutlon process. The 'helping* and 'hindering' forces are then ranked 
from moat to least Important. After goals have been decided, action can be 
taken to either strengthen 'helping' forces or to weaken hindering forces, or 
both. 



To our knowledge the technique has not been used In curriculum 
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development In TAFE. The method, In analysing change situations, has the 
advantages of offering a systematic approach to problem solution by teasing 
out the helping/hindering forces and Identifying the points at which change 
should be applied. It therefore provides the strategy for change, e.g. 
strengthen specific helping forces, weaken specific hindering forces. 

While not a curriculum research method in the formal sense, it does appear 
to offer a useful technique, if used following an NGT which identifies the 
problem areas, to tease apart the helping/hindering forces which bear on the 
Identified problem. In this application, the method would have the advantage 
over a solution-generating form of NGT (as distinct from a problem -identifying 
form) of providing greater opportunities for the group to share and interact on 
perceptions and ideas, whilst retaining the 'hitch-hiking' (building on the ideas 
of others) advantages of NGT. 

Sample 

Groups of 4-8 persons. 
Applications 

NGT is mainly a tool for providing directions in situations of complexity and 
resistance to change, e.g. a planning group wishing to foster the uptake of an 
innovation (technical or educational) might use the method to analyse 
implementation difficulties and adopt a strategy for change. Similarly, a group 
wishing to introduce a needed change might use the method to show where 
effort (leverage) might be applied to best effect. 

Critical Incident Technique 

Uses 

To obtain a description of behaviours (completed action sequences) that are 
'critical 1 , in terms of either success or failure, to the performance of a task or 
critical gaps in knowledge, i.e. 'never do X whey Y is happening.' 

Data collection method 

'Critical incidents' data can be collected by observers (e.g. researchers, 
supervisors) or the job holder. Data can include: what led up to the incident; 
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what actions were effective (or ineffective); perceived consequences of the 
action; and whether the consequences were within the control of the job 
holder. Collection procedures can include recall, observation, interviews or 
questionnaires. 

Sample 

Job holders. 
Sample size 

This depends on the problem but sample size can be reduced through 
stratification with fewer cases treated in greater detail. 

Advantages 

Since the data are observable behaviour, the technique is said to maximise 
objectivity while reducing subjective components of inference and 
interpretation. That is, the method draws out factual information of the 'how 
to/how not to 1 type, rather than the 'I think ...' type. 

Applications 

The technique is suitable for eliciting practical information about working 
with equipment, processes or people. It can be used to decide weights of 
emphasis in curriculum design and assessment decisions (what content is 
'critical 1 ). It has wide application in studying social skills needed in technical 
and further education. 

The concept of 'critical incidents' (i.e. do's and dont's) can be used in other 
research methods, e.g. an NGT focused on a part of the work of an 
occupation. In student questionnaires, questions can be asked on (1) what 
aspects of the course were critical and positive in helping the student to 
perform key task sequences successfully, or (2) what parts of the course were 
irrelevant, or (3) what task were you required to do but were unprepared for? 
This information would, of course, need to be set into perspective by analysis 
of the student's background such as type of work performed by employer. 
Critical incidents collected across a pool of respondents could be the basis of 
an industry or student questionnaire. 

29 
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Duration and cost 



As yet, there are no case study applications in TAFE of this method. The 
method can, however, be time consuming in data collection and analysis but 
appears worthwhile in terms of the practical relevance of the data to 
successful work performance. 
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CHAPTER 2 
SEARCHING 

2.1 EXPLORATORY (QUALITATIVE) METHODS 
Introduction 

A central problem in curriculum design in TAFE is to decide what should be 
the broad thrust of a program: the aims, main content areas, weights of 
emphasis to be placed on subjects and course structure (subject sequencing). 
Some research procedures, especially those which gather detail about tasks 
performed (task lists or inventories) provide the base data on work functions 
and tasks performed, but usually do not provide, from the research itself, 
guidelines as to how this data will be organised into a finished curriculum. 
The problem of translating occupational data into a curriculum is described 
next. 

In attempting to explain the problem of deciding what to include or exclude 
from a course, we will use the analogy of the 'line of best-fit' used in 
statistical regression analysis. Just as regression analysis seeks to plot a line 
of best fit through a pattern of information (a scatter of data points), so too, 
does curriculum development seek to find a best-fit pathway which embraces 
specific information such as the main work functions or the tasks performed in 
the occupation (i.e. data which describe what is done) or competencies required 
for work in an occupation (i.e. data which interpret work functions in terms of 
knowledge and skills required for effective job performance). The analogy is 
depicted graphically in Figure 2.1 below, which represents the field of 
knowledge required within an occupation as circles which, for purposes of 
illustration, have been labelled 'concept-skill clusters', i.e. the concepts (theory 
knowledge or knowledge 'how 1 ) which guide the practical skills (knowledge 
'what') needed to execute a task. In Figure 2.1 these concept-skill clusters 
have been positioned, arbitrarily, according to two dimensions: breadth of 
coverage and complexity. The 'pathway of best fit' in Figure 2.1, represents 
what is to be included in a course. 



22 

31 



breadth 
of 

coverage 



Pathway of 
'best-fit' for 
deciding course aims, 
scope and sequencing. 




complexity 



Fig. 2. 1 Representation of curriculum content decisions 
as a 'best-fit 1 pathway through a subset of the total 
knowledge within an occupation 

Curriculum development which works from the ground up by attempting to 
organise a mass of detail about tasks, sub-tasks, and the like, into course aims, 
structure and content, often leaves the placement of the pathway through the 
total field of knowledge to the data analysis and curriculum design stages* 
Where such decisions take place without benefit of research-based guides as to 
how the f base data 1 will be structured into a course, the decisions about course 
aims, structure and content become subjective (based on personal opinion or 
'hidden agendas 1 ) and on arguments of the kind: 'you have to put this in the 
course ... you can't teach this without that 1 . In other words the decisions are 
subject to personal opinion. Such comments may be sound but how can this be 
known from a research basis? An answer to the subjectivity problem when 
translating 'base data' into course aims, appears to lie in blending ground-up 
methods which give basic detail on work functions, with overview-down methods 
which explore the larger questions of occupational structure and change in 
order to decide the main elements of course aims, content and structure. This 
theme, the need to understand how work in an occupation is structured and is 
changing in order to guide decisions about course aims, scope and subject 
sequencing (finding the pathway of best fit) will now be taken up by discussing 
some of the key concepts which underpin exploratory research. 
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Key concepts in exploratory research 

The aim of exploratory research in vocational and further education is to 
achieve an understanding of the main features of the occupation or activity 
being studied with a view to deciding the larger questions of curriculum design: 
course aims, course structure and how course content is to be organised. 

Sampling methods which might be used during exploratory research include 
•snowball 1 sampling, used when the researcher is circling around in information 
networks - literally like a snowball gathering size as it moves, and typical 
applications 1 sampling - the selection of cases to provide a cross-section of 
typical applications of work, equipment and processes used. This type of 
sampling is sometimes married to matrix sampling where representative types 
might form one dimension of the matrix and hierarchical levels within the work 
place might form the other dimension. 

The thrust of exploratory research is, therefore, to provide insights into the 
following: 

. Scope of the program; 

. Weights of emphasis to be given to segments of the 
program; 

. Depth of coverage within segments; 

. The order in which subjects or units should be 
sequenced so that students will acquire usable skills 
(as defined within the occupation) or marketable skills 
(to secure a place within an occupation, or gain 
promotion), which are appropriate to the stage of 
education (e.g. first, second, third year or advanced). 

Good exploratory research begins with good research questions. A research 
question describes your aims in the research phase: what you want to find 
out. Research questions which guide exploratory research are usually purposely 
defined in broad terms since their aim is to provide the overall picture, not 
the minutiae of task elements. Research questions are usually divided into 
finer questions (e.g. as in an exploratory interview) and are used as the 
researcher's organising perspectives, guiding and focusing the data search and 
data interpretation stages. Some examples of research questions are: 
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• What gaps exist in educational provision within an 
occupation (which could be filled by TAFE course X)? 

• In what ways is technological change altering the 
pattern of work skills in the occupation? 

• How is the occupation structured from the point of 
view of designing relevant curriculum? What career paths 
are open? What does a student need to know to enter 
the occupation? What knowledge is required at more 
advanced levels? What upward or sideways career paths 
are open? 

. What should be the central (broad) outcomes of the 
level of course being designed/reviewed? 

. What constraints exist within TAFE which might pose 
program implementation difficulties? 

Research questions used in occupational research should ideally span the 
individual level (educational outcomes) and the occupational/social context level 
(the wider forces which shape immediate knowledge needs or future needs 
which the educational program should anticipate). In moving back and forth 
between these two perspectives (individual and occupational/social context), 
exploratory research seeks to relate student needs to an understanding of the 
way in which work in an occupation is organised, or is likely to be organised 
by the time graduates will enter the occupation. Exploratory research 
therefore borrows from psychological (individual) and sociological (wider social 
context) perspectives. The attention given to the sociological or wider 
perspective ensures that exploratory research produces the necessary broad 
picture of how work in an occupation is organised. 

One method of examining the way an occupation is structured is to look, in 
broad terms, at the kinds of work functions which employees are required to 
perform upon entry to the occupation and then to trace likely career paths 
(sideways or upward) in order to assess the goals (outcomes) of a broad-based 
educational program. However, looking at the way an occupation is arranged 
in terms of existing work functions can miss important changes which are 
beginning to take place within an occupation, e.g due to the uptake of new 
technologies. New technologies 'de-skill 1 by replacing old skills with a 
technological innovation. However, in this process, a demand for new skills is 
created. 

The aim of exploratory research is to detect patterns or common themes 
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(e.g. emerging skill requirements) but also to look for 'disconfirming instances' 
which may imply that the emerging pattern does not tell the full story. Good 
exploratory research, therefore, looks for cracks or flaws (disjunctures) in the 
emerging data pattern. These disjunctures then become leads for further 
exploration. 

For example, exploratory research into occupational educational need in a 
TAFE Post-Trade course' (Ahderson, 1985) showed the patterns of knowledge 
required for effective, advanced level job performance but revealed a 
disjuncture (flaw); the industry did not believe that TAFE could deliver an 
effective educational program because TAFE lacked high technology 
equipment. The school rectified the equipment problem before seeking 
accreditation of the course, avoiding an initial failure which would have 
jeopardised the future of the course. This example demonstrates how 
exploratory research can generate an important understanding of how initial 
appearances, in this case a clear view of the patterns in educational need, can 
mask vital information, in this case doubt within industry that TAFE could 
deliver proper training. This problem, the 'pattern level' of the data obscuring 
vital disconfirming information, is also present wherever consensus is used as 
the main argument for a particular line of action, a point which has relevance 
to a number of the curriculum research methods described in later sections, 
which seek to establish, by various and sometimes subtle means, consensus 
about curriculum content and aims. This issue is taken up again in Chapter 4. 

Exploratory research can be very productive in a short time, especially if 
based on good research questions applied in probing interviews with informed 
personnel and observation of work performed. A useful sampling strategy is to 
select 'typical applications' of work in an occupation which provides a 
cross-sectional view of the occupation according to sampling criteria which 
evolve as knowledge of the occupational area expands during the research. 
That is, the sampling cases and the research questions are reviewed and 
expanded as needed during data collection. This is in contrast to the 'classical' 
approach to research which takes a straight line (linear) form: 1. Literature 
search. 2. Hypothesis fprmulation. 3. Data collection to test hypotheses. 

The aim of exploratory research is to achieve an understanding of the 
occupation/social context at the broad level. The only hypothesis in 
exploratory research may be the hypothesis that, if the right questions can be 
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found, the answers will follow. Another example: A short exploratory research 
project was conducted which examined knowledge needs in an occupation 
affected by technological change (Anderson, 1985). Since technological change 
was known to be a key factor, a 'typical applications' sample (n = 4) was 
selected for the exploratory phase which reflected technological uptake at four 
levels (1 case per level), extending from 'high use of technology' through to 
'mainly old technology'. This sampling strategy was intended to throw light on 
a problem often encountered in TAFE curriculum development projects: the 
need to know what older generations of equipment are still in use, in order to 
decide the scope of the educational provision. 

In exploratory research, sampling is continued until an adequate 
cross-sectional view of an occupation is obtained. In the project mentioned 
above, the sampling was terminated at the four cases because; (1) The four 
cases provided an adequate cross-sectional view in the eyes of the research 
team and the sampling categories were confirmed by respondents as covering 
the main ways in which the occupation was divided. Had confirmation of the 
logic behind the sample not been obtained, additional cases would have been 
added to obtain a cross-section and the process of adding cases would be 
continued until the data began to be repetitious. This demonstrates a principle 
in exploratory research: the exploration should continue until an adequate 
impression is obtained of the phenomenon under study, sacrificing detail rather 
than breadth, if resources are limited. (2) The intention was to use the results 
of the exploratory research to design a questionnaire to be administered to a 
20 percent random sample of the occupation to amplify the impressions of the 
exploratory phase. 

While exploratory research can provide qualitative information sufficient for 
course development to proceed, for this to be successful either the syllabus 
writers must have a good technical grasp of the area and/or be able to draw 
upon existing materials such as technical manuals or other publications for 
detail once the overall aims/outcomes are known. Exploratory research 
(including the yield from preliminary investigation) provides an excellent basis 
for designing a questionnaire because it can provide a set of major questions 
which can be explored in the questionnaire in more detail. Insights about the 
way that duties are arranged in an occupation, or insights about the forces for 
change acting within or upon an occupation can be quantified and these then 
used to sort responses to specific knowledge and skills questions according to 
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occupational structural characteristics. The questionnaire results can show 
differences between knowledge/skills for, say high technology firms versus older 
generation technology firms. 

A considerable body of knowledge has been developed about how to build a 
course from the ground up using a variety of questionnaire methods which 
obtain very detailed information about job tasks. (See Gael, 1983, on use of 
the Work Performance Survey System [WPSS]; Banks et al. 1983 - Job 
Components Inventory [JCI]; Youngman, 1978 - Position Analysis Questionnaire 
[PAQ] and Goode, 1981 - Comprehensive Data Analysis Program [CODAP] to 
mention a few). Approaches which work upward from detail about work 
functions to course structure and aims, invariably fall short in providing 
information about broader educational needs of students such as learning how 
to learn in order to obtain a sufficient breadth of knowledge to facilitate 
career mobility. 

The strength of good exploratory research lies in its potential to pick out 
the broader dimensions which will affect curriculum design: to work from the 
top down, from the general to the specific, by identifying the broader 
outcomes for educational provision. For example, an exploratory investigation 
of short courses for Graphic Arts camera operators (in NSW) showed that the 
ability to work unsupervised was a central requirement of employers. This 
became the central goal of the course provision and was written into the 
course aims. 

In the conduct of exploratory research and indeed, research in general, care 
needs to be taken when looking for patterns, to look also for any flaws 
(disconfirming data) which should be followed up with further exploration. 

Good exploratory research is something of an art form. Its aim is to 
explore the dimensions of the subject matter being investigated, both to inform 
the researcher of the questions which need to be asked and to obtain data 
about the industry being studied, which will have a direct application to the 
curriculum to be de . -loped, especially at the level of course structure, aims 
and main content. Exploratory research involves being in the right place at 
the right time, with the right question and places a premium on the knack of 
being able to synthesise (join data), yet make leaps (look beyond data). It 
needs interpersonal skills to engage people in co-operative efforts, to 'keep the 
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information coming 1 , yet to avoid being deceived by initial appearances 
(patterns). It is a form of research which sweeps from the individual to the 
wider occupational/social context and involves a continual process of probing, 
questioning and reflection. 

Exploratory research requires the ability to engage 'key informants' (people 
with either broad knowledge or limited though detailed knowledge of the area 
under investigation) in open-ended discussions which explore a set of research 
questions; adding more questions, if needed, until an understanding sufficient to 
decide the broad thrust of the course (structure, aims, main content) is 
obtained. 

Exploratory work, though at times difficult to quantify, can be used to focus 
quantitative methods on those areas where such methods appear to work best. 
Moreover, it can be useful in setting the results from quantitative research 
into a wider context e.g. by relating fine-grained detail on tasks and 
competencies, to an understanding of occupational structure and change. Since 
public education invariably lags behind the emergence of new skills, it is 
possible, for those industries or services which are undergoing dramatic change, 
that a good blend of qualitative and quantitative data, obtained and used 
quickly, could lead to the development of an educational program which 
contributes to the design of the job itself, rather than merely reacting to an 
existing design. 

In summary, the aims of exploratory research are: 

(a) To obtain 'broad-brush' insights into questions of scope, weights of 
emphasis (where is breadth of coverage and where is depth of coverage likely 
to be important) and sequencing of learning (what needs to be placed early in 
a program given two requirements: The needs of the occupation, and of the 
needs, in knowledge acquisition, for certain concepts/skills to be acquired 
before others. For example, problem-solving skill is knowledge based: certain 
knowledge must be acquired before problem-solving skills can materialise). 

(b) To obtain an understanding of how work in an occupation is organised 
and the ways in which this may change in the near and distant future. 

(c) To obtain an understanding of what a student may need in relation to 
the motivation for self-training and life-long learning. 
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We have presented an approach to conducting exploratory research using a 
sample of 'typical* firms to give a cross-sectional view of work in an 
occupation, using focused interviews plus observation of work done. The 
principles of this method can also he applied to the study of skill masters (see 
Appendix C). 

2.2 THE SEARCH CONFERENCE TECHNIOUE 

2.2.1 Background to the Search Conference technique 

The Search Conference has been used widely both overseas and in Australia 
in the government and private sector as a tool to achieve participatory 
problem solving or planning. In the educational domain, Search Conference 
techniques have not yet been widely used for curriculum design and review, but 
for identifying educational issues, establishing educational policy and priorities 
and developing plans for solving educational problems. 

The Search Conference has promise for undertaking curriculum research 
activities. Advantages claimed for the method are: 

. Involvement of all appropriate stakeholders in the 
search process. 

. Capacity to generate commitment to the search outcomes 
and consequent enhanced prospects of implementation. 

. Futures (or forward-looking) orientation of the process 
which seems to be particularly suitable for the planning 
of vocational educational programs. 

. Philosophical consistency with androgogy (the education of 
adults) and the reliance of the Search Conference on the 
traditions of 'action research 1 as distinct from pedagogy. 

. Attention to the values, beliefs and power relationships of 
the participants and the educational decision-making 
environment (Crombie, 1985). 

2.312. Tfr? philosophy of the Search Conference 

The Search Conference is \ planing process based on a perception of the 
world as comprising a turbulci t uironment - one in which at any particular 
time, people and institutions 'ia* * ^oals, but that these goals will undergo 
frequent changes. The meti.-v ^>umes that the values and beliefs of people 
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are less changeable. As a method of participatory planning therefore, 'search' is 
a method which begins the process of generating action plans by engaging 
participants in an exchange of values and beliefs. 

Crombie (1985) presents a view of the Search Conference within the 
tradition and framework of 'action research' and from the point of view of an 
adult educator. The Search Conference, viewed from this perspective, appears 
to be particularly suitable to curriculum research undertakings in TAFE. 

The Search Conference is a process built upon an acknowledgement of a 
number of requirements for achieving effective social change and development, 
including the willing participation of members of the social system undergoing 
change, leadership of the change process, and relevant expertise for the 
framing of choices (Crombie, 1985). Because of this the Search Conference 
contrasts favourably with some of the traditional 'expert' based approaches 
often used in TAFE curriculum decision making. For these reasons it is 
claimed, 'search' affords a greater potential for generating commitment to 
outcomes and enhanced prospects of implementation (e.g. by teachers). 

As a tool for the achievement of social change and development, the search 
methodology is designed to generate co-operative group activity aimed at: 
finding solutions to practical problems, contributing to the body of knowledge 
on the area under focus, and heing a learning process for the participants. 

Crombie (1985) identifies the Search Conference process, with its attention 
to values, beliefs and power relationships, as a political process. As such, he 
urges search group leaders to be clear about their own values and to accept 
responsibility for communicating these values to others. It is evident therefore 
that people's values - both the participants and leaders - or rather the 
exposition of these values by individuals and the exchange of value perceptions 
in the group process, is a cornerstone of the search philosophy. The 
•past-present-future' framework for expressing and exchanging values and views 
of desirable futures by participants reflects the forward-moving (planning ahead) 
and divergent thinking characteristic of the Search Conference method. It is 
not a method which relies upon the common preference for focusing on the 
here and now, the known, or the objective, but is one that encourages the 
opening up of new horizons which can then of course be assessed in terms of 
present constraints. 



m Aa tMPte of tfca Bsareh Conference method 



According to Crombte (1909) the Soorch Conforonce It • methodology for 
participatory planning. It It • systematically structured and managed process 
whereby groups can s ea rch future possible environments for e more deslroble 
future for themselves, and generate e stretegy for ochlevlng it. It Is a 
purposeful but realistic process, In which 'deslroble' futures ere Identified on 
the basis of an exploration of shared humen values, end weighed against an 
•valuation of the past, and an assessment of present constraints and 
opportunities. In lu basic structure the process moves from the generation of 
information and Ideas (brainstorming - divergent end leterel thinking), through 
synthesis end analysts of these (evaluating and priority setting), towards action 
planning. 

The S ea rch Conference begins end ends with plenary (whole group) sessions, 
the structure end purposes of which can be reasonably well determined 
beforehand. The Initial plenary session must both begin to build group 
cohesion, and embark upon the teak which Is usually described In general 
terms, e.g. *Wetcome to this Seerch Conference on future directions for ...' 
This Is likely to entail: 

. Breaking Into small tasz groups' of three or four 
people In order to Introduce one another, to Investigate 
the resources In the group, end to clarify perception! of 

. A briefing on the task and the p ro p osed overall structure 
of the conference from the group facilitator. 

. An Initial brainstorming session on the forces that are 
shaping the future environment. In some cases this session 
may fee eddreesed to the pest rether then to the future - 
•where have we come from?* 

The mtterlal generated In these and all subsequent sessions Is recorded on flip 
tHsrt* t» butchers paper, which will comprise the raw material for the final 
report. 

The final plenary session Is for reporting from small groups on priorities and 
strategies, end for collective action planning - who is to do what, when, with 
what resources? This usually Includes the selection of some sort of steering 
group to finalise and distribute the report. 
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Aside from these two basic elements, the structure of each Search 

Conference (refer to Figure 2.1) is likely to vary somewhat according to the 

task and the practitioner. There are, however, a number of common or typical 

steps and it Is possible to characterise these, though not to assign them to any 
particular order: 

FUTURE SCAN 

Brainstorming session in plenary session to generate data and value 
perspectives on the trends, forces, events that are shaping the future 
environment. 

DESIRABLE AND PROBABLE FUTURES 

The product of the FUTURE SCAN is taken as a starting point by small 
groups, formulating from it the most desirable future and the most probable. 

PRIORITIZING 

As an alternative to the above, the group may be asked to Identify the 6-10 
trends or changes that they consider will be of most importance, and to spell 
out the influences that can be expected. 

INTERNAL SCAN 

In some cases it might be appropriate to run a brainstorming session in the 
plenary session on changes and trends that are occurring inside the organisation 
or school or college and shaping its future. 

HISTORICAL RECONSTRUCTION 

In some cases a rewinding of the clock to see 'where have we come from? 1 
can be useful. This leads naturally to a consideration of what should be 
preserved, and what should be modified or dropped. 

STRENGTHS AND WEAKNESSES 

This can be an alternative to, or extension of, the INTERNAL SCAN. Its 
function is a stocktaking of the present situation in terms of 'what are we 
good at? 1 
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ISSUES AND PRIORITIES 



While most of the key issues and options will have been identified in one or 
other of the preceding sessions the task here is to consolidate the issues that 
confront the group, and place priorities on them. 

ACTION PLANNING 

Small groups select priority issues to develop action plans for reporting back 
to the final plenary session as key recommendations of the conference. 

FINAL PLENARY SESSION 

Final report on action plans, and consolidation of these. Discussion of 
implementation procedures. 

In the simplest terms, a Search Conference aims to assist people towards 
effective decisions and actions to be taken now, by clarifying where or what 
they want to be in the future, and taking account of the claims of the past. 



FUTURE 



PRESENT 



PAST 



Where do we want to get to? 



Where are we now? 



Where have we come from? 



TIME 



Figure 2.1 Basic elements of a Search Conference 



2.2+4 Planning and conducting a Search Conference 

The following guidelines have been identified by some search practitioners as 
a useful aid to the planning and conduct of effective conferences. 

Conference venue 

An appropriate conference (workshop) venue should: 
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. be accessible to participants 

• have overnight accommodation facilities if required 

• have adequate parking facilities 

• have wall space for attaching butcher's paper sheets 

• have sufficient chairs and tables 

• preferably comprise one large room 

• have a separate area for refreshments 

• be private and relatively free of outside noise. 

The venue should be inspected to check its suitability. 
Selection of participants 

Participants should be chosen carefully and with due courtesy and protocol 
by considering: 

• who are the stakeholders in the matter to be searched 

• can the number of participants be contained within the 
range of 15-35 (ideally)? 

• what is the conference (workshop) about? 

• why hold a Search Conference rather than using some 
other strategy? 

• do the likely participants have relevant experience, 
knowledge and interest? 

• can a sense of commitment be anticipated from 
each participant? 

• what should be the balance of stakeholders within the 
selected group (e.g. teachers, industry representatives, 
practitioners, students, administrators)? Establishing 

the correct balance of stakeholders is vital to the success 
of the outcome. 

Participants should be telephoned and their attendance confirmed by letter. 
Preliminary contact with participants 

In preparing for a Search Conference it is important to spend up to two 
weeks (for 30 participants) in attending to the following preliminary activities. 
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Major stakeholders should be interviewed (whether they are participating or not) 
to: identify the issues that are likely to arise in the workshop; identify the 
circumstances of the present situation (e.g. what the current curriculum is and 
how it is delivered); identify what problems and constraints people are aware 
of; and explain the search workshop process and begin to share values, 
philosophy and to gain initial commitment to the process. 

Conference agenda 

The agenda for the workshop should be designed carefully, taking into 
account the preliminary work described above. Although the search process is 
basically the same format for each workshop, each agenda should be uniquely 
determined to suit each group and the problem to be tackled. (Time 
constraints may require the contraction of two or more steps into one). It 
may be advisable to convey to participants an indication of timing on each 
stage of the process, since some participants are particularly time oriented. 
Ideally, a search workshop will be conducted over a period of between 10 and 
20 hours, spread out over two days. 

Sample agendas for short Search Conferences ('mini-search conferences 
spanning an evening and the next day as opposed to the more usual three day 
format) are provided in Table 2.1 and in Appendix & The example given in 
Appendix B shows the final format adopted for a 'mini-search' for the review 
of the Graphic Reproduction Trade course (Anderson, 1986). The example in 
Table 2.1 was drawn from a Search Conference conducted by Dr Alastair 
Crombie in co-operation with the Curriculum Services Directorate of NSW 
TAFE (now Directorate of Studies). The study was conducted as one of a 
number of research strategies what were adopted for a review of the NSW 
TAFE Pastrycooking Trade Course (Brady, 1984). It was attended by 17 
participants comprising six industry representatives, one union representative, 
six TAFE teaching staff, three students (one from each stage of the existing 
pastrycooking trade course) and one TAFE curriculum development officer. 

The Search Conference workshop was facilitated by Dr Crombie as workshop 
manager - he was assisted by a co-facilitator. Two workshop observers were 
present. One of the observers recorded the conference on viueotape. 

The conference was conducted over two days (and one evening session) and 
lasted for about 12 hours. 
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Workshop materials 

The Search Conference facilitator should be properly prepared and armed 
with the requisite materials to avoid delays and interruption to the process. 
Materials might include: 

. large sheets of paper (butcher's paper) 

. overhead projector and transparency materials 

• masking tape 

. wall space for affixing the butcher's paper sheets 
when completed 

• appropriate handouts (such as agendas or guides on 
writing action plans) 

. ease's, boards, marking pens, rame tags. 
Preparation of workshop room 

Arrive at the venue early to allow time for preparation including: 

• distribution of handouts and name tags 

• location of boards, paper, pens 

. preparation of refreshment area 

. arrangement of furniture and the room to maximise group 
sharing and group unity. See Figure 2.2. 




preferable 



not preferable 





tables and chairs 
for small groups 



□ 



o □ i 

□ □ v 



or for large groups (where unavoidable) 



permanent tables 

seating 

6-8 people 



chairs only 



Figure 2.2 Seating arrangement for a Search Conference 
Workshop introduction 

The introduction to the workshop should be low-key but efficient. The 
blowing points are worthy of note: 

. introduce and welcome people as they arrive; 

. start on time; 

. thank people for attending; 

. introduce self and allow for introductions of others; 

. explain why workshop is being held and > arranged 
or authorised it; 

. key speakers (of the expert or autiwity type) should be 
avoided as they can be counter productive. (If protocol 
insists that such a person must say something then let it 
be restricted to a welcome to participants.) 

orkshop beginning 

The workshop should commence with a recording of participants 1 expectations 
i butchers paper. Using one or two scribes as necessary, record answers to 
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the questions: 



. why are you here? 

. what do you expect from the workshop? (Include 
facilitator's expectations but do this as part 
of the group). 

As well, ensure that the group feels comfortable. 

When recording expectations: 



. do not prolong the activity unduly, but do tolerate 
some silences; 

. do not attempt to qualify or summarise participants' 
contributions; 

. record what is said in full (but use keywords or 
interpretations if the speaker rambles on); 

. once recorded, leave the sheets on the wall and proceed 
straight on via 'well that's quite a job* or 'O.K. let's get 
on with it' or such like, in order that the expectation will 
be that recorded expectations will be met. 

Workshop process guide 

Explain the role of the facilitator, the agenda and search process, stressing 
that the search process is based on a philosophy and perception of the world 
comprising: 



. a turbulent social environment; 

. goals change frequently, whereas values are more 
long lasting; 

. planning is based on the shared values of the 
participants not on the views of 'experts'; 

. be open and honest and encourage the group to 
be likewise. 

. value all opinions equally regardless of status 
or position. Promote an awareness in the group of 
the narrowing-in process or the Search Conference 
which is based on sharing values about: the world 
or Australia; narrowing to the subsystem under 
examination, e.g. education -> TAFE -> a particular 
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TAFE course; narrow to particular problems, issues, 
constraints; narrow to particular action plans. 



A picture of how the Search Conference moves from the general to the 
particular is given below. 




action to — - includes curriculum 
follow development 

A 

This can include working groups 
reporting back within a set time 



Figure 2.3 The Search Conference process 



Stress that the differences between 'search* and traditional planning processes is 
that the latter often founder when the external social environment changes 
(MacDonald, 1984). Stress that the facilitator's role is one of attending to the 
process, not giving answers which is the group's job. When brainstorming (the 
process of generating ideas in a group setting): 

. tolerate silences (but be prepared to break them by 
prompting a discussion area not yet considered, such 
as a factor believed to be important in course development 
for the area; 
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• be flexible with time limitations; 



• encourage ideas to come as fast as they are generated 
(using two assistants to write them on the sheets of 
paper, if necessary); 

. do not foster argument, except for clarification; rather 
encourage the opposite viewpoint or a qualification 
and record that as well; 

. check with the speaker (at least by eye contact) 
that the idea has been recorded accurately; 

. hear everything and value all contributions - if one 
idea out of three coming at once is missed, stop the 
group and find out what was miss d and record it; 

. discourage speeches (advise the group th^t this is 
a rule) by avoiding eye contact wh they occur or 
looking impatient, but listen carefully nevertheless and 
record some key words to let the person know the 
idea is still valued; 

• help the participant who is having difficulty in 
crystallising a thought by being patient, quietening the 
room, suggesting a key word; 

. do not agree or disagree with contributor's views; 

• keep the session moving, but be prepared to stop when 
the group has had enough. 

Note that brainstorming is an ice-breaking activity. It can develop group 
cohesiveness and good humour. Importantly, it introduces ideas which the small 
groups will consider in depth. It is important to encourage a comprehensive 
treatment of the topic and record in brief clear wordings. 

When examining 'desirable futures' in small groups (6-8 people): 



• indicate to participants that this is where the group 
can work towards an agreed shared set of values; 

. try to eliminate any constraints which might limit 
the range of ideas generated; 

• each group should strive for consensus; 

. try to devise strategies allowing more time to those 
small groups who need it; 

. ensure each small group has a recorder (scribe) but 
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not a chairperson; 



• the facilitator should not join any group but should 
'float 1 asking probing questions if needed. 

During plenary (report-back) sessions after small group sessions: 



• restrict questions during report-back - questions or 

discussion can occur at the end of the report-back session; 

. encourage the speaker to refer to the butcher's paper 
recordings for his/her group; 

. encourage speaker to be brief and not dwell on points 
made by other groups; 

. if the small group session was focused on planning, 
then seek plenary consensus on an issue by issue basis 
(and be prepared to send groups away to do it again). 



Identifying issues 

When individual participants are posing personal issues to be considered: 



. provide participants with a format guide to Titing 

down their issues so that there is a common format, such 
as: (1) What is the issue? (2) Why is it important? 

• encourage a sense of ongoing commitment. 

Whether issues are identified by individuals or small groups, the facilitator 
should promote group ownership and agreement so it is preferable to allow the 
plenary group to identify, cluster and prioritize issues. (If preliminary research 
has identified issues which during the workshop seem to be overlooked by the 
group, then say so and ask why?) 

Setting up planning/action groups 

. when forming action groups to undertake work on specific 
issues 

>*• 

- base the groups on the issues identified and prioritized 
by the whole group and not on one person's perceptions; 

- the composition of the groups should be voluntary; 

- expect a firm commitment to ongoing action; 
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- encourage participants to join action groups where the 
issue is of concern to them and where they feel they 
can contribute; 

- allow 'floaters' (although in time most participants will 
settle into just one group). 

- when action plans have been prepared by interest groups, 
gain plenary acceptance of the action plans. Also: 

- encourage groups to use a common format for setting out 
their action plans, e.g. what is the issue, why is it 
important and how can something be done? (Seek a range of 
alternative strategies in priority order); 

- encourage interest groups to test their plans on other 
groups or to brainstorm other groups 

- when each action group reports back to the plenary 
session, encourage the assembly to constructively 
criticise the action plan - if appropriate the group can 
re-design their action plan, and if necessary ask for 
the action plan to be re-written. 

. At all stages of the workshop, especially at the stage of 
discussing action plans, it is important for the 
facilitator to be consistent and to assist participants in 
being consistent. The facilitator should keep referring back 
to previous . group and plenary reports and point to*' areas 
of inconsistency if these exist. The primary role of the 
facilitator here is to ensure that agreed, shared values from 
the earlier part of the workshop permeate the action plans 
that are accepted. 

. In planning for the follow-up work of action groups, the 
groups themselves should be encouraged in the ownership 
of their plans until the issue (or problem) is resolved 
or the task is completed. This will almost; certainly require 
further meetings or at least the establishment of some form 
of network. The facilitator should ensure that the issue is 
finalised, but should not need to be the prime-mover to 
achieve this. 

Workshop closure 

Achieve closure of the workshop by reviewing the expectations of 
participants stated at the beginning and summarising ongoing action. Thank 
participants for their contributions. Compile a workshop report and distribute 
this as soon as possible after the workshop, being sure to include copies of all 
the butcher's paper sheets and action plans. 
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2.2.5 Utility of the method to curriculum development in TAFE 

This section offers some comments on the usefulness of the Search 
Conference method in a TAFE curriculum development context. The 
comments, at this stage, are impressionistic. 

Duration of the method. From the point after which preliminary research 
activities have been completed, a search approach would appear to take up 
about four working weeks of the organiser's time. This period would account 
for preparation (two weeks), conduct of workshop (one week) and reporting (one 
week). 

Cost of the method. The major TAFE costs associated with the conduct of 
a Search Conference would include the work time of the conference organiser 
(a teacher or curriculum officer) for four weeks; the fee of the outside Search 
Conference workshop manager, if this expertise is not readily available inside 
the organisation; the conference work time of participating teachers and other 
officers; the hiring fee for appropriate conference venue; the costs of provision 
of meals and refreshments for participants; the costs of travel and 
accommodation of conference participants; and printing and postage costs 
associated with preparation and distribution of conference report. 

Expertise required to use the method. The most important component of 
expertise required to use a Search Conference method is the skill and 
experience of the search workshop manager - the nature of this expertise is 
more fully covered in Anderson and Jones (1986), The facilitation of curriculum 
research workshops in TAFE. 

The Search Conference organiser (whether a teacher, curriculum officer or 
administrator) would need to have or be able to draw upon curriculum 
development expertise. Oftentimes the organiser may choose to work as a 
co-workshop manager (or co-facilitator) with the specialist workshop manager. 

The teaching and industry expertise that would be required in using the 
method, would generally be provided by the Search Conference participants 
themselves - from the TAFE teaching area and the industry area for which the 
search study is being conducted. 

Human resources required to use the method. The number and selection of 
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people to be involved in the use of a Search Conference method is determined 
largely by the diversity of workshop participants required to adequately 
represent the views of the key stakeholders in the activity. Typically, ;he 
range of people involved might include: a workshop manager, a curriculum 
officer, between five and ten teachers, two to four students, five to ten 
industry representatives and between two and four TAFE administrators. 

Hardware requirements of the method. As mentioned previously, the 
material requirements for the conduct of a Search Conference would include 
the workshop venue comprising one large room with ample wall space and a 
number of accompanying smaller rooms for small group work, and butcher's 
paper, writing pans, easels (on which to place two sheets of butcher's paper 
side by side for writing down the groups comments) and an overhead 
projector. Other items were listed in the previous section titled 'Workshop 
materials'. 

Capacity for blending the method with other approaches. Because of its 
relatively short duration, and low cost, the Search Conference method is 
particularly suitable for use as a part of a broader research strategy. If used 
in this way, it would seem appropriate to use the search approach during the 
early part of the overall research phase of a curriculum project. The capacity 
of the Search Conference for examining the context of a curriculum project 
and identifying the likely problems to be faced, as well as its 'futures' 
orientation give it considerable power as a method to be employed as part of 
a broader research strategy. 

Acceptability of the method. Some applications of the method in its short 
or 'mini-search' form have been made (Brady, 1984; Anderson, 1986). The 
method does appear to hold promise for setting the broad directions for 
technical education courses and is well worth trying, especially in situations 
where major technological change is affecting an industry. It is essential to 
achieve a full representation of stakeholders to ensure that the data are 
representative. 

The acceptability of the method in a TAFE Authority will depend to some 
extent on the curriculum research tradition established in that authority. 
Where curriculum researchers have pursued research styles from the positivist 
scientific tradition, then there may be some ideological concern about adopting, 
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•vw m • component of a research strategy, . method which is as 
participatory, future.-orl.nted and qu.lit.tlve .. . Se.rch Conference. 
Comply, however, It I. not unlikely th.t TAFE .dmlnl.tr.tor. end/or 
consumers will be keen to employ methods such as setrch because of the 
relatively snort duration of its execution. Needless to say, TAFE curriculum 
"searchers, in designing research strategies, and conducting research, should be 
concerned .bout striking .n appropriate bal.nce between elements such as- 
espediency of the method, reliability/validity of the method, and acceptability 
of the method. 

Notwithstanding these Introductory remarks, It appears to the writers that 
the search method has certain positive attributes: 

* ■^^ ^ ^ , E h,r, "L the re$earch proceM <" an active 

ill"f Ji?if C,ve . t0 earn,ng comn »ltment by those 
pay w ho art instrumental in the Implementation of the 
rMMrcn outcomes; 

* lnnSmi7 , .,S!, l .?? JOr ,trtehol < ter « - «udents, teachers, 
Ind^y^admlnUtrators. etc In the research process In an 

. II Is cooperative, participative and based on the shared 
of par tlclpant. which are likely to be more 
enduring than the stated goals of people; 

* totures-orlented which is particularly suitable to 
if £2uJl TV 1 "!* • Imed 10 meet tne requirements 
envlr^en^ "* vocat,on • , «•■« m a changing 

' i !J b fV n f Pfrt'cularly suitable process for 

^ S ln *w, of lhat wou,d ,ead curriculum 
ZSfSIJLS 0 • bte t0 . dw « , mlne an appropriate 
course philosophy/rationale, course alms and a 
course structure. 

The method seems to have the following drawbacks: 

* "J^L™!? 1 I" TAFE for curriculum 

E^JJEf ", iU 2 mUy M • radical departure 

from traditional and entrenched research methods; 

' i L^JJTt !£ P * r *JSl! mf !2 # capMUy to comprehend the 
JSJ^2S2L2L meth0d and 0008 comprehended to embrace 

£ S^^L^J! VaUd rat,ona,e for Participating in 
tne process Mid being committed to its outcomes; 

. while it appears to be a useful method to adopt as an 
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initial curriculum research method, it does not possess 
a mechanism which is a unique aspect of the method itself, 
for defining the detailed curriculum content/objectives 
which would ordinarily be the major outcome of a typical 
TAFE curriculum research activity. For developing detailed 
curriculum content/objectives, it is likely that an 
additional research and/or analysis step would need to be 
used as an adjunct to the method. In Chapter 6 we suggest 
the use of the DACUM method for detailing curriculum content 
and objectives. However, we see no reason why the Search 
Conference in its full three day format cannot be focused 
onto the fine detail of subject content and objectives. 

The extent to which the method generates results which are reliable descriptors 
of the occupational or target area being studied, as well as of the needs of 
students, is dependent upon how well the participants represent the industry 
and the clients of the educational program. Since the search method requires 
the composition of the group participating in t..e Search Conference to be one 
reflecting the interests of all stakeholder groups, particularly industry, teachers 
and students, it is evident that the method does have the capacity to generate 
a useful picture of how technological change is altering the skill and knowledge 
requirements of an occupation. This capacity would presumably be enhanced 
by increasing the numbers, representativeness and expertise of those important 
data sources. The method may also reveal conflict between the parties 
interested in controlling the curriculum. 

However, the ideal search workshop, limited as it is to about 35 
participants, does not appear to provide as much potential for providing these 
reliable descriptors as a number of the more traditional quantitative research 
methods. (The point that should be borne in mind here though, is that the 
search method, by its very nature, is constructed on a rationale which affords 
a trade-off between large sample sizes and representativeness on the one hand, 
and specially selected interest, expertise, shared values and commitment on the 
other hand.) 

Utility of the method's results 

Circumstantial utility 

Some search practitioners claim that there are no particular curriculum 
research circumstances where the search method would not be usable (Crombie, 
1984, MacDonald, 1984). It has been argued that the search method is 
appropriate to curriculum research in TAFE across a range of study or 
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occupational areas, across all course levels (trade, technician, short specific 
purpose, personal development), across different student populations, and in 
different regions or communities. It would seem that the search method might 
be particularly suitable to projects where one or more of the following 
characteristics exist: 

. search is serving as a beginning research strategy 
for a curriculum project; 

. the circumstances are such that new courses are being 
developed; 

. the occupational areas are those where there is rapid 
technological change; 

. the study areas are those where the existing curriculum 

has not been examined or reviewed thoroughly for some time; 

. the circumstances are such that there are likely to be 
diverse views and different perceptions among stakeholder 
groups of the state of the existing curriculum and of what 
changes should take place. 

Curriculum development utility 

In respect of generating data that are required for the purposes of 
developing assessment methods and other teaching/learning and delivery 
strategies and resources, it would appear that the search method again has no 
in-built facility for achieving these outcomes. If these outcomes are desired, 
then search would probably require supplementing with another research and/or 
analysis tool. These kinds of tasks could then be undertaken by one or more 
of the action groups formed at the conclusion of the Search Conference. 

Quantity of data collected 

In broad terms, a search method will not generate masses of data - mainly 
because this is not the central purpose of search. Rather, a search approach 
will result in general and qualitative data which is valuable in setting the 
scene and exposing and resolving issues which will be pertinent to the 
curriculum development activity to follow. 

Capacity of the method to forecast 

The Search Conference method is admirably suited to delivering data about 
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perceived future educational/training needs. Indeed, its whole structure is 
tailored to ensure that interested, concerned and expert people have an 
opportunity to share views on the future events that are likely to be important 
in stating educational needs for the present and future* 

Capacity of the method to induce change 

Theoretically, the search method has a great capacity to facilitate the 
achievement of desired curriculum changes. The extent of this method's 
capacity to support curriculum change can be appreciated when one considers 
the well-documented fact that people who are crucial to the implementation of 
curriculum changes (such as teachers) need to be actively involved in planning 
the curriculum change. This is the case with the search method. 

As well, the innate dependence of the search method on sharing of values 
between major stakeholders and the derived sense of ownership and 
commitment to its outcomes, means that the search process does not ignore, as 
so many traditional research methods do, the important part played by people's 
attitudes (values and beliefs) in securing any kind of change. This has 
certainly been shown to be the case with curriculum change in TAFE. The 
attitudes and expectations of teachers especially are a very substantial variable 
in the equation of successfully securing curriculum change in TAFE. 

As already noted, however, 'search' is not yet fully tested. But it does have 
promise particularly for illuminating the larger questions of: 

. thrust (i.e. course aims); 

. scope (what's in or out?); 

. weights of emphasis (where do these lie?); 

. sequencing (what before what?) 

However, obtaining full representation from the cross-section of stakeholders is 
vital since last minute cancellations can seriously affect the quality of the 
conference output. Considerable attention must be given to following-up those 
invited. Those who cancel at the last minute should be replaced. For this it 
would be useful if replacements were picked at the time the original sample 
was selected. 
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TABLE 2.1 

PROGRAM FOR A SEARCH CONFERENCE FOR 
PASTRYCOOKING TRADE COURSE (NSW TAFE) 



DAY ONE: COMMENCING LATE AFTERNOON 

5.00 p.m. Brief context setting introduction including 
explanation of broad purpose of workshop, by TAFE curriculum 
officer responsible for conducting the review of the course. 

5.0b p.m. Workshop manager commences proceedings by: 1. 
Ensuring participants were not unhappy about the presence of 
video equipment being used to record the proceedings for research 
purposes; 2. explaining relative small size of group - some 
industry representatives who were invited to attend were unable 
to do so; 3. providing an indication, in broad terms, of the 
timetable of the workshop; 4. providing an indication of the 
overall composition of the group of participants; 5. inviting 
participants to self-select into four groups (mixing with people 
not known) in order to share with the group ir'ormation on (i) 
who am I? (ii) where do I work and what's my role? (iii) what 
experiences do I bring to the workshop? The workshop manager 
emphasises the importance of item 5 above, because the premise 
of the process is sharing and using all experiences. 6. Manager 
asks each group to nominate one member to lead a report-back 
session in the plenary session at 5.45 p.m. 

5.10 p.m. Plenary report-back. Nominated member from each 
group reports on backgrounds and experiences of each member of 
group. During this session, the workshop manager introduces self 
and his co-worker. Manager enquires whether there is any concern 
about the presence of observers. 

6.00 p.m. Conference briefing by workshop manager. Briefing 
comprises: 1. Review of conference timetable. 2. Manager provides 
overview of conference via: (a) provision of scope of 'futures 1 
thinking for conference by suggesting a '1988 onwards' 
perspective, (b) provision of general guideline that we need to 
think what industry will be like then, work back from there and 
then determine what TAFE needs to do. 3. Reference to prepared 
schematic plan for Search Conference a** below: 
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A. Course review 

l 

C. Where have / B. Future scan 

we come v. 
from? ^ 




D, Present strengths V \ & Participants' 

and weaknesses \ experiences 

F. Future directions in 
pastrycooking training 

4. In explanation of the plan above: (A) The TAFE curriculum 
officer provided the overview of 'where have we come from/ Item 
(B) comprises the 'crystal-ball' gazing phase of the 
conference. Item (C) comprises the 'wind back the clock' 

phase of the conference by reference to the wisdom of the past, 
(D) Comprises the 'here and now' component - what are we good at? 
What are the gaps? Item (E) acknowledges that our experiences are 
an overriding input throughout the conference. These experiences 
are shared throughout. Item (F) addresses the question of how to 
design a better pastrycooking course, 

5. Manager states the goal as he perceives it, viz, to at least 
arrive at a broad statement of what a pastrycook will need to 
know from 1988 onwards, 

6. Questions and expressions of concern from participants are 
invited and discussed. These included: (A) Unrepresentativeness 
of the group - some participants felt certain important 
stakeholders had been overlooked (the ones identified were from 
specialised sections of the industry), (B) A teacher participant 
posited the view that industry people should determine their 
training needs. Manager responds by indicating that industry 
representatives will be involved in that process later in the 
workshop, (C) A participant queried the logic of proceeding from 
futures to past to present, in that order. Manager responded by 
reference to de Bono (lateral thinking) who stresses the 
importance of using our creative imagination about what 

is desirable before constraining or measuring this against the 
past or present difficulties, 

6.30 p.m. Dinner break. Participants attend a prepared buffet 
style dinner provided in a nearby room, 

7,30 p.m. Future scan session. Manager advises on timetable 
for session and reminds participants how this session fits into 
the scheme for conference. He advises that the whole group will 
first spend 30 minutes in a lateral thinking stage and then in 
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three separate groups will put judgements (values) on the 
suggestions recorded. 

Manager advises of three rules to be observed in the half hour 
plenary session. These are: 1. No speech making. 2. Short, sharp 
telegram-like suggestions. 3. Conflicting points of view are 
acceptable; these will not be debated or argued. 

Manager reminds participants to think of the trends, major 
changes of which there seems to be evidence of occurring over the 
next 5 to 10 years. He also reminds participants to concentrate 
on the forces impinging on the industry (to stay outside the 
circle) as shown below: 



forces 
impinging 




Manager records contributions carefully. When two or more 
participants speak at once, he records one and returns to ensure 
the other(s) have not been missed. During this session, manager 
reminds participants to focus on what they see happening in the 
next 5 to 10 years - not what they would Mke to see. After 
about 20 to 30 contributions have been recorded, manager begins 
to round off session by observing that 'we seem to have the main 
ones ... have we done justice to this stage?; sach one could be 
further detailed, but this can be done in ^malier groups.' 
Several more contributions and clarifications are made and 
recorded. 

Manager invites participants to divide into three groups with a 
mix of participants according to background. He provides groups 
with task guidelines as follows: 1. Sift through all the 
contributions recorded and strive to reach some agreement on the 
major trends. Pick out six to eight items as most 
significant. 2. Begin to spell out the implications each one has 
for TAFE training using the following framework: 

Major trends Implications for training 

(a) 

(b) ~ 

(c) ZIZ^ZIZI^ZII 

etc. 

3. Spend 30 minutes and report back at 9.00 p.m. 

8.30 p.m. Three groups meet in separate nearby rooms and 
attei.d to the task set for them. 
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9.00 p.m. Plenary reports. (Nominated member from each group 
reports back.) At the end of each report-back, manager asks for 
any additional comments from group members or clarifications from 
other, participants*. »• 

9.35 p.m. Future scan close. Workshop manager reminds 
participants of the schedule for tomorrow. 

DAY TWO: FULL DAY SESSION 

9.00 a.m. Where have we come from? 1 Workshop manager recaps 
on purpose of conference and focuses on the next task of looking 
backwards. He encourages participants to look back at the major 
developments that have occurred in the industry and in TAFE 
training. A chronology begins to emerge beginning around 1926. 
Manager reminds participants during the course of this session 
that exact historical accuracy is not so important, however 
'we 1 should strive to get an impression of important changes that 
have taken place. 

Workshop manager records comments and seeks clarification as an 
ongoing process. He sometimes asks 'why did this event occur?' 
Manager rounds off session with 'have we got all the significant 
ones for now?' h few more contributions are made and recorded. 

10.30 aum. Morning tea. 

11.00 a.m. 'Present strengths and weaknesses'. Manager 
invites participants to spend 10 to 15 minutes considering the 
present strengths and weaknesses of current training both in TAFE 
and on-the-job. 

He advises that this session should be conducted in a large group 
setting and could simply be achieved by participants making 
contributions according to the following format: 

Strengths Weaknesses 



Manager emphasises the valuable contributions students should be 
able to make to this session. After recording 10 to 15 entries in 
both columns, he asks 'have we got all the important ones?' 
Again a few more contributions are made. 

11.20 p.m. Development of course objectives. Workshop manager 
then advises that we have arrived at the core of the problem, 
i.e. to begin to build a new course. After a quick recapitulation 
- future - past - present, he advises participants that they are 
not required to design the entire course in all its curriculum 
detail, but should first focus on: 1. The philosophy of the 
course. 2. The structure/organisation of the course. 3. The 
content areas of the course. 



etc. 
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Manager suggests that for 1. above, each group should work to 
produce a statement of 'course objectives' which might serve to 
go in a course publicity brochure - a statement that would inform 
the reader about what a graduate would gain from the course. 
These should take into account the three domains of skills, 
knowledge and attitudes to be learned. For 2. above, he suggests 
that each group should try to produce an outline of the overall 
organisation of the course such as 



For 3. above, manager suggests a broad identification of the 
content areas of the course. Manager observes that to complete 
all of these in the time available is a major task and asks 
participants do they feel it is too much? 

Participants indicate that the task is possible - in any case it 
can be commenced and this will be helpful to the whole review 
process. Manager suggests that for this task two larger 
groups be formed so as to limit the diversity of conclusions at 
this stage. 

Finally, the manager advises that the outcomes of these sessions 
will comprise the major part of the conference report. Reporting 
back from each group will take place at about 2.00 p.m. 

11.45 p.m. Two groups begin work on development of course 
objectives. 

12.30 p.m. LUNCH BREAK 

1.00 p.m. Two groups continue work on course objectives and 
structure of course. 

2.10 p.m. Plenary report back. 

Nominated member for each group reports back. For example 
Group 1: Provides a fairly general statement of course aims. 
Leads on to a discussion of course structure with recommended 
durations for units of study. 

Manager asks plenary group to develop the aims and structures 
proposed, taking into account the list of strengths and 
weaknesses generated earlier. Discussion concludes that the list 
of strengths and weaknesses will probably be more useful in the 
detailed curriculum development phase of the review. 



D- □ - 



a series of subjects 



or units in sequence, or 




i.e. a core of study followed by 
various specialist studies 
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A discussion about the advantages and disadvantages of modular 
courses for pastrycooking ensued. Manager conducted a 'straw 
poll 1 on the question of suitability of modules for 
pastrycooking. He added that the decision of such a poll was 
significant because it reflected the view of people who 'cared, 
were experienced and who had participated in this conference. 
•This was the key to success of the conference outcomes.' 

A teacher contribution at this stage indicated that to determine 
the curriculum details for a revised pastrycooking course would 
require the identification of which content areas of the 
existing course were irrelevant. He observed that such decisions 
could probably not be made at this conference. 

3.05 p.m. 'Where do we go from here? 1 

Workshop manager addresses the mechanism that now needs to be set 
in train to put into practice the outcomes of the workshop. He 
urges that the group should now take responsibility for what 
happens next. He suggests that three participants should meet to 
write up the papers generated by the workshop and this should be 
done as soon as possible. This draft would then be coordinated 
and recommendations based on these formulated to contribute to a 
final report. Such a small coordinating group serves as 
•trustees 1 for the whole group. The report should receive wide 
circulation. Manager also suggests that this group might care to 
consider how the revised course will be maintained. 

The coordinating group is nominated - the curriculum officer; a 
teacher; and two industry representatives. 

4.00 p.m. Conference close. The workshop manager praises 
participants for their contributions and observes that he feels 
the outcomes were worthwhile. 



2.3 THE DELPHI METHOD 
Background and rationale 

The Delphi method is a decision analysis procedure which has been 
extensively used in forecasting. It was developed in the 1950s as a way of 
finding the most reliable means of gaining the consensus of a group of experts 
about the likely pattern of future events concerning American national 
defence. Delphi is a general methodology rather than a fixed procedure. One 
of the advantages of Delphi f is its flexibility to circumstance and degree 1 
(Ulshack, 1983:114). 

The philosophical underpinnings of the Delphi method are covered in some 



detail by Linstone and Turoff (1975). The Delphi method assumes that society 
is directed towards goals and that these goals are the product of the actions 
of interest groups. Delphi does not necessarily ask what these interests are 
but asks what likely events are foreseeable. Delphi provides a means for 
finding out the areas of agreement and disagreement about the pattern of 
likely or desirable future events. By use of a series of questionnairs it draws 
this information from a panel of people who possess a position in society which 
provides a vantage point for viewing the interplay of a variety of forces. 
Because of the complex nature of these forces, no single person is in a 
position to provide an answer. But, according to the method, a carefully 
selected group, joined by a process which gives each access to the others' 
perceptions, can give a reliable estimate of future trends in a specified area. 
According to Ulshack 'Delphi may productively be used in need identification 
and setting training priorities for specific subjects' (1983:113). 

Overview of Delphi 

Its essential features are: 



. It searches the views of a panel of selected experts, 
users, resource controllers or administrators, in 
relation to an issue or problem and then aggregates the 
judgements to identify likely trends. 

• The points of view elicited are summarised and returned 
to respondents, a process which continues until consensus 
is approached or achieved. 

. The method, in its conventional form, does not require 

participants to come together or to be known to each other. 

A Delphi search can focus on expected major developments in a variety of 
fields (Bunning and Turoff, 1979:174), including trends in relation to new 
materials most likely to be taken up within an industry within a specified time 
frame (Enzer, 1975). It can provide a series of scenarios, future-tense 
descriptions of events that have not yet occurred with Delphi panelists judging 
the relative likelihood and ultimate desirability of the scenario event; or it can 
be used for developing probable cause lists for specific performance problems 
on specific service skills (Zemke and Kramlinger, 1982:150). Delphi can be used 
to clarify a problem located in the present or the future or to identify the 
essential components of a solution. 
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The process involves a series of feedback loops in which a monitor group 
poses a problem or question, gathers initial reactions from expert participants, 
collates or summarises the information which is then sent back to the original 
group, enabling each to view the other's statements before commenting again 
on the issue or problem. An important advantage of the method is that 'input 
gained as part of one step of the process is shared as part of the next step' 
(Rasp, 1976:151). The number of feedback loops needed is determined by the 
emergence of consensus from the group. Usually two or three rounds are 
sufficient though five rounds have been used (Delbecq et al. 1975:103). The 
number of panelists can be a handful or a hundred or more. 

The critical factor in the success of a Delphi is that 'each participant must 
be genuinely interested in solving the problem' (Zemke and Kramlinger, 1982). 
Compared with the Nominal Group Technique (NGT), Delphi requires the least 
amount of time for participants, but takes longer in terms of calendar time (up 
to 45 days). Delphi, like NGT, prevents domination by a few individuals 
(Delbecq et al. 1975:83). 

Linstone and Turoff (1975) describe two versions of Delphi, the conventional 
method and the conference method. Both versions involve administration of a 
questionnaire. In the conventional method a monitor team designs a 
questionnaire which is distributed to the respondent group or panel. The panel 
does not meet, nor are they identified to each other. The monitor team 
summarises the results and, based on the results, develops a new questionnaire 
so that the panel is given at least one opportunity to re-evaluate its original 
answers based on examination of the group response. 

In the conference method, the Delphi process is applied while the group is 
assembled. Again, a monitor group prepares and administers a questionnaire, 
e.g. on an issue which has arisen. The very short time available for analysis 
usually means that the monitor group is replaced to a large degree by a 
computer which has been programmed to carry out the compilation of the 
group results. 

Both forms of Delphi include four phases. The first phase explores the 
Delphi question and may use open-ended questions which can be analysed into 
categories. The second round can use closed questions, possibly with rating 
scales depending on the topic. In the second round phase an understanding is 
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reached of how the group views the issue in terms of dimensions such as 
■importance 1 , 'desirability' or 'feasibility' and includes agreement, disagreement 
and the meaning given to any relevant terms. The third phase entails an 
exploration of disagreement to bring out the underlying reasons and possibly to 
explore these reasons. The process is terminated when consensus is reached or 
the patterns of agreement or disagreement are reasonably clearly defined. 

Delphi can also be used in a way that approximate? survey research, 'where 
the variables are already developed and concern is only with refinement and 
movement towards consensus concerning the relative importance of individual 
variables' (Delbecq et al. (1975:90). 

An example of the utility of Delphi in forecasting changes within an industry 
was provided by Enzer (1975:199-209) who applied the technique to the plastics 
industry in the USA. His study examined the material needs and resource 
allocations within the industry, and was based on the assumption that the types 
of plastics produced in the future would be determined more by what was 
desired (and pursued) than what was technologically possible at the time. This 
is an important point in favour of Delphi applied to an industry study. It can 
show how the likely actions of industry decision makers will exert pressure for 
certain aspects of current or emerging technology to be taken up. In contrast, 
a study which only examines what is technologically feasible in an industry 
(e.g. by surveying the literature on technological developments) may not 
indicate the likely direction which industry decision makers will take in relation 
to what they perceive as the likely market forces bearing on the industry. 
Delphi can illustrate 'how current policies and future possibilities may work 
together to form the desirable future' (Bunning, 1979:175). Data on likely 
industry trends can be very useful data for educational resource and curriculum 
development planning. 

Enzer's (1975) application of Delphi to the plastics industry in the USA is 
summarised below: 

Plastics were categorised into materials and properties (engineering, general 
purpose, glass fibre, foam, and non-plastics). Specific plastics within each 
category were listed. The Delphi panel was asked to: (1) review the materials 
and properties presented, indicating where they thought changes which would 
significantly affect the widespread use of the material were likely to occur 



within the next 15 years; and (2) describe the anticipated properties of new 
materials which they thought were likely to evolve and gain widespread use by 
the specified target date. In both of these steps, the panel was asked to 
describe the new chemical, and physical or other technological developments 
that they believed would lead to the creation of the new material. 

These inputs from the first Delphi round were used to prepare a three-part 
questionnaire for the final round of interrogation. These parts were (1) a 
summary of the panel's assessment of anticipated changes in existing material 
properties, such as weather resistance and impact strength; (2) a listing of both 
plastic and non-plastic materials with the nature of the anticipated major 
changes described and (3) a list of new materials expected to be available by 
the target date and a description of their properties, e.g. price, processability, 
tensile strength, etc. All thee parts of the questionnaire were open-ended in 
that respondents could add any additional items or comment on any item. 
Respondents were asked to rate the plastics in terms of whether they were 
'outstanding in the property indicated; among the best in performance 
available', or gave 'acceptable performance in this property; still suitable in 
most cases', or were 'not acceptable if indicated property is important to 
intended use.' 

The results of the first round Delphi were summarised and returned to the 
panel. Next, participants were asked if they agreed or disagreed with each 
summarised statement. In addition to changes suggested in existing materials, 
other materials were suggested by the panel as prospects for widespread use by 
the target date. These were submitted to the panel and their comments were 
sought on anticipated properties. By asking panelists to estimate production 
trends, Enzer (1975) was able to identify the likely growth points within the 
industry for certain types of plastics. An application of Delphi along the lines 
outlined above which included processes as well as materials, would provide a 
good basis for curriculum and resource allocation within an industry. 

A study of the steel and ferroalloy industries (Goldstein, 1975:210-226) used 
the method of presenting a set of forecasting assumptions (future scenarios) 
which asked respondents to extend into the future a series of trend lines on 
production graphs and, in so doing, to state the key assumptions underlying the 
trend line extensions and the major uncertainties which they believed would 
affect these trends. Thus, the question sought to bring out the thinking behind 
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the forecasting estimates m well m the estimates themselves. 
Goats end time 

Delbecq et al.(l97S) give the following estimate of the time token to 
complete e Delphi process. 

'I. Develop the Delphi question (0.5 day) 

2. Select and contact likely respondent* (2 days) 

3. Select sample site (0.5 dey) 

4. Develop and teat first questionnaire (10 days) 

5. Analyse questionnaire (0.5 day) 

6. Develop and teat second questionnaire (II days) 

7. Analyse questionnaire (I day) 

ft. Develop and teat third questionnaire (II days) 
ft. Analyse questionnaire (I day) 
10. Prepare and send out report (7 days) 

TOTAL TIME: About 45 days.' 

and conducting a Delphi proe m 



Delphi Involves three groups of people; (I) the decision makers who will 
utilise the outcomes, i.e. those who have commissioned the study, (2) the 
professional team who will carry out the study and (3) the respondents or panel 
whftM *?!dfements are being sought* It is important to return the results of 
e*tfi raind to participants promptly. Bunnlng and Turoff (1979:180) recommend 
a 'ten-day turn around time.' They advise that respondents who do not return 
their questionnaires on time should be sent a new questionnaire Immediately, 
followed by a tactful .e'tilnder. The third round questionnaire is reported to 
be the most difficult for respondents to complete and suffers the highest 
attrition rate (Bunnlng and Turoff, 1979:180). 
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steps ta the method 

I. Develop the Delphi question(s) 

The first step In developing the Delphi questlon(s) is tc define the objectives 
of the study, the type of Information which Is required, and the types of 
respondents needed to ensure a cross-sectional view. The aim of the Delphi 
question Is to develop a •broad range of Input 9 (Rasp, 1976:154). Burning and 
Turoff advise that the number of questions should be limited to as few as 
possible and the questionnaire should provide only limited space (perhaps three 
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which participants can respond (1979:181), Examples of Delphi questions Include: 
•Think about the professional nurse delivering outpatient (ambulatory) health 
care. What Is the professional nurse responsible for today that such a person 
should not be generally responsible for ten years from now? In each case, 
provide a brief example, possibly from your own experience, that leads you to 
believe this Is an Important and desirable change?* The responses to this 
question were later returned to respondents in the following form with 
participants asked to rank the top seven Items (see Delbecq et ah 
1975:149-171): 



responsibilities Comment 



fn ten years, nurses in an 
outpatient/ambulatory setting 
will be responsible for: 

(List of Items generated) 



In a study of the needs of a school district (Ellenstown), Rasp used the 
following Delphi approach (1976:152-3). The questionnaire asked respondents to: 
•Think In terms of what conditions are likely to be present during the decade 
1975 to 1985 and what the results of the Ellenstown schools ought to be In 
view of these conditions. Specific statements would be more helpful than 
general ones. 1 

As a result of the experiences provided by the Ellenstown School District, 
students should: 

Know 

Be able to 

Feel 

Ellenstown School District should: 

Increase 

Maintain 

Reduce 

Develop _ 

The second round questionnaire sought ratings as follows: 
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low ability high ability 

1 2 3 4 5 Be able to read and understand the 

newspaper. 

In the third round questionnaire, the same layout was used as the example 
immediately above, but the priority most frequently mentioned by other 
respondents was shown by enclosing one of the numbers in a box. Respondents 
where asked to circle their choice. This information enabled the consensus to 
be calculated and also a minority or dissenting report could be prepared for 
each item. The final data gave priorities, consensus percentages and dissenting 
opinions. 

2. Select and contact respondents 

It is vital that respondents feel personally involved in the problem of 
concern to the researchers, have pertinent information to share, are motivated 
to see the process through and value the type of information which will be 
produced. Once the general characteristics of the desired respondents are 
identified, the final sample is usually selected by a nomination process in which 
key people in the target group or groups nominate those who would be in a 
position to provide the necessary information. Random sampling may also be 
used. Zemke and Kramlinger (1982:150) caution that 'a single pane t; st 
performing in a perfunctory manner can create distortions'. Delbecq et al. 
(1975:88) advise that potential respondents should be contacted by telephone or 
in person by 'someone whom the respondent respects 1 who will explain the 
objectives, the nature of the panel, the obligations of respondents and the 
length of time the process takes. 'Only after a detailed and personal 
introduction will most respondents participate fully in the Delphi process' 
(Delbecq et al. 1975:88). 

3. Select sample size 

With a homogeneous group of people, 10-15 participants can be enough. But 
in cases were a number of target groups are involved, the sample might run to 
several hundred people. Delbecq et al. (1975:89) report that with 
homogeneous groups few new ideas are generated once the sample group 
exceeds thirty well-chosen participants. The greater the number of 
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participants, of course, the more time consuming is the analysis. 
4. Questionnaire analysis 

Round 1. The purpose of the analysis of the first-round questionnaire is to 
arrive at a mutually exclusive but exhaustive set of categories which 
accurately conveys the meaning which respondents attempted to communicate 
in their answers to the first questionnaire. Bunning and Turoff (1979:177) 
suggest the following rating scales for trend, desirability and needs 
assessment. 



'the likelihood or feasibility of this trend developing 
will be: 

4 - almost certain 

3 - probable 

2 - possible 

1 - improbable 

0 - virtually impossible. 

the desirability of this trend occurring will be: 

4 - very great 

3 - great 

2 - moderate 

1 - slight 

0 - no or none 

Training needs priority: 

1 - high priority 

2 - above average priority 

3 - average priority 

4 - below average priority 

5 - lowest priority' 

Round 2. The questionnaire for the second round asks participants to review 
the items identified in the first round. The aim is to identify the areas of 
agreement and disagreement, the items requiring clarification and to obtain an 
early understanding of priorities emerging. Participants are asked to review 
the list of items, commenting, asking questions or arguing in favour of against 
items; and to select the ten items they feel are most important, assigning a 
value of 10 to the most important item and 1 to the least important item 
(Delbecq et al, 1975). 

The analysis should record the frequency of comments on items, and 
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summarise comments made about items. This pro\ides preliminary indication of 
the priorities which the panel has given to particular items. 



A form of second round questionnaire used by Bunning and Turoff (1979:177) 
presented respondents with a list of items and a five point scale on which was 
shown the 'modal consensus 1 (circled): 

'Emerging or future markets to consider in promoting 
(Brand X) for use in product manufacture are: 



Round 3. The final questionnaire permits respondents to review prior 
responses and express their individual judgments as to the importance of each 
item. 

Suitability for use in TAFE 

An important use of Delphi would appear to be in forecasting short-term 
(0-5 years) and medium-term (6-10 years) trends within an occupation, in order 
to provide additional information for educational resource planning and 
curriculum development. The utility for curriculum development of accurate 
trend information on materials, processes and equipment, is obvious. But 
Delphi has other uses. Rasp (1976:150) notes that Delphi has been used to 
predict developments in teacher education, educational administration and 
federal funding. The conference form of Delphi can be used to explore and 
identify the central themes in an issue which arises in a curriculum 
development workshop or seminar. The great advantage of the process is that 
it links participants so that each has access to the thinki of the other before 
forming a final opinion on a complex issue. Delphi appears to offer an 
excellent 'warm-up 1 procedure when it precedes a workshop. Delphi with one 
round of feedback would make a good introduction to a Search Conference or 
a DACUM or NGT workshop. In such an application, the clarification of 
agreement or disagreement could take place during the workshop before an 
attempt to identify or prioritize essential elements. A full Delphi followed by 



your response 



consensus 



1. Solar energy development 1 2 3 




Reason 
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a Search Conference in which the Delphi results were included in the future 
scan section would overcome one of the uncertainties about the Search 
Conference method: the ability of the participants to read the future in 



Advantages and disadvantages 
Advantages claimed for Delphi include: 



• It provides a framework by which individuals who are 
distant in space can be linked. Since travel time and 
cost is eliminated, the method can include people who 
would not normally be able to participate in a group setting. 

• Participants have access to the pooled data before giving 
final comments or priority rankings. 

• Participants remain anonymous. 

• The method describes what key decision makers believe is 
likely to happen, given what is technologically possible 
and indicates areas where industry activity is likely to 
exert pressure for technological advances arising, for 
example, from likely market demands. 

. The method is forward-looking and provides a process 
which keeps respondents focusing on trends. 

• The methods provides time for thinking and reflection, 
including reflection on first thoughts in the light of 
the ideas of others. 

• It is claimed that the method tends to minimise 
psychological communication barriers, such as leadership, 
hidden agendas, personality conflicts and group pressures 
for conformity. 

• It provides each participant with equal opportunities 
for influences. 

• It provides precise documentation. 

Disadvantages/limitations 

Delphi is largely supported and legitimised by assumptions rather than 
research findings. Two assu^ ;t^ns are: f (l) if participants agree and by 
agreeing, move closer to a central position or consensus, then the resulting 
data is more believable; (2) that am nymous responses such as those generated 
by the Delphi are more likely fo lead to reasonable and objective input than 



depth. 
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are the activities of interpersonal conferencing' (Rasp, 1976:151). Delphi data is 
limited to subjective judgments. However, there are times when this is Hie 
only data available. 

The development of the second round questionnaire can be difficult. Rasp 
(1976:154) notes that 'uneasiness' can occur amongst users at this . -age. 
Moreover, 'the pull toward consensus is strong, frequently stronger :,an a 
position of verified accuracy' (Rasp, 1976:152). In other words, the pressure to 
achieve consensus may bias respondents' replies. 

Another problem is that some future events are unpredictable. As Mark 
Twain said: 'Actually, February is a very bad month in which to predict the 
future. So is March, April, May...!'. Other disadvantages (Bunning and Turoff, 
1979:179) include: under or over-estimating the likelihood of future 
developments; there is an inherent difficulty in imagining the future even after 
key events have been assumed; it is often difficult to distinguish desires from 
forecasts; it is often difficult to define and integrate cross imp-ts ;;r.ong 
specific forecasts; and important possibilities are often overlooked. 0>he: 
difficulties include that of stating expectations in the crispness of language and 
the precision of judgement more often associated with technological changes. 
Participants do ,iot like to start, so to speak, with a blank sheet of paper. 
There is a heavy expenditure of time in completing the various oue^onnaires 
and 'the method relies on participants who have much stamina 1 (Finch and 
Crunkilton, 1979:132). 

Conclusions 

Delphi has the advantage that it collects, and organises judgements i ? a 
systematic manner. It builds a picture of where consensus lies and it shows 
dissent (Rasp, 1976:155). The method 'avoids the pitfalls of public group 
discussion: group pressures, an unwillingness to abandon publicly rtated opinions, 
and the effects that individuals have on groups' (Bunning and Turr-ff, 1979:178). 

In a comparison between the Nominal Group Technique and Delphi it was 
found that the Delphi group provided more alternatives to a problem situation, 
and more innovative ideas (Zemke and Kramlinger, 1982:151). 

A stance strongly critical of Delphi was taken by Weaver (1971). His 
criticisms apply to forecasting methods in general and so have applicability to 
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the Search Conference method as well as Delphi. Weaver argues that 'In order 
to effectively evaluate forecasting methods one must be able to test the 
plausibility of the forecast which results' (1971:267). To test plausibility, one 
must test the reasonableness of the arguments which support the forecast. 
Delphi skirts this issue by selecting expert consensus or argument as the 
measure of the forecast. 'But consensus alone is not a sufficient condition for 
arguing that a forecast (s plausible and convincing' (Weaver, 1971:269). Weaver 
points out that it is not possible to precisely identify the Dephi findings which 
'reflect rational judgment as opposed to feeling of desirability' (1971:268). It is 
possible, as Weaver notes, that the data base available to social science 
forecasting is shifting and more unreliable than technological data. Social 
science forecasts 'are more likely biased by personal values than technological 
forecasts' (1971:269). This suggests that Delphi, in curriculum research 
applications, should concentrate on identifying those aspects of change in an 
occupation which the panel sees as apparent now (even if the signals are only 
comparatively weak) and that the time frame should be more like 'now to five 
years time' rather than the more distant and more speculative 5-10 years; and 
that Delphi should concentrate on technological aspects such as material, 
processes and equipment. 
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CHAPTER 3 

ANALYSING THE HERE AND NOW 1 OF AN OCCUPATION 



3.1 THE NOMINAL GROUP TECHNIQUE (NO T) 
Background to NGT 

The nominal group technique grew out of psychological studies in the USA in 
the 1950s into ways of improving group interaction on problem-solving tasks, 
when it was found that groups in which individuals worked alone, that is, were 
a group in name only, hence 'nominal 1 groups, produced more ideas under 
'brainstorming 9 rules (generating ideas without criticism) than groups in which a 
problem was talked over (Shaw, 1971:72). 

The Nominal Group Technique is a structured process of group 
'brainstorming 9 which facilitates group-based decision making when applied to 
certain types of planning problems. It is a process strictly controlled by the 
leader: verbal interaction is restricted, to a large extent, to leader-individual 
dialogue and discussion within the group is actively discouraged by the leader. 
The degree of leader control, unless sensitively handled, can pose problems in 
applying the method. 

The Nominal Group Technique can be used to define a problem or to 
generate solutions. The method has two distinctive features: (1) a 
'brainstorming 9 stage in which members respond to a carefully designed question 
by writing ideas silently and independently, and (2) a process of mathematically 
deriving the group's consensus as to which are the important dimensions of the 
problem or the most salient solutions. 

The method has been widely used as an aid to solving organisational 
development problems and has been successfully applied to curriculum 
development in TAFE. Although the appropriate group size is 8-10 people, a 
number of NGT groups can be run simultaneously, or the procedure can be 
used on a number of different target groups. The method is described by 
Ofteil (1981:58) as a powerful and cost-efficient method of data collection 
where views are sought about the efficiency or effectiveness of particular 
programs or the likely effects of changes in particular programs. Before 
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discussing the reasons behind NGT and the method in detail, we will present a 
summary of its main steps. 

The main steps in an NGT workshop are: 
• Explanation of the method. 

. Presentation of the 'NGT question: which will serve to 
focus the brainstorming efforts of rhe group. 

. Silent generation of ideas in writing by the group. 
(See Note below.) 

. Statement of ideas by each member in turn (round-robin 
listing) until all ideas are listed on a Hip chart 
visible to the group. 

. Discussion of ideas is limited to clarification only. 

. The listed items are merged, simplified or organised, 
as needed and with the agreement of the group. Again, 
discussion is limited to clarification. 

. Preliminary voting to establish priority of items. 

. Discussion of items - additions and merging. 

. Final vote. 

NOTE: These 'ideas' refer to either elements of the 
problem or solutions depending on whether the 
NGT is attempting to find what is the problem 
or, if this is known, to find solutions. Sometimes 
these two stages are covered in one NGT. 

Rationale for NGT 

NGT is based on the idea that one of the critical process characteristics 
facilitating creativity is the separation of problems from solutions (Delbecq et 
al. 1975:22). Group processes which 'separate ideation (problem identification) 
from evaluation (solution getting) are superior to group processes which 
combine them' due to the human tendency to seek solutions before the problem 
is fully understood (Delbecq et al. 1975:22). This claim is supported by 
Hegarty: 'The structured nature of the technique helps a group to attain 
superior performance in fact finding and idea generation when identifying 
problems as a starting point in the decision-making process' (1975:33). Delbecq 
at al. (1975:27) suggest that disagreement between group members can either 
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lead to hard feelings or to creative decision making depending on how it is 
controlled. Hard feelings can be generated by allowing disagreements to 
become personal or be smoothed over by humor or withdrawal. Creative 
problem solving, they argue, can be achieved by separating persons from 
problems, and attacking the problem, not the person (1975:27). 

Delbecq et al. (1975:9) make a number of claims for the method. These 
are discussed in turn. 

'1. It assures different processes for each phase of creativity 
(fact finding and evaluation). 

2. It balances participation among members. 

3. It incorporates mathematical voting techniques in the 
aggregation of the group judgment/ 

Claim 1. In the fact-finding stage NGT generates data about the problem or, 
alternatively, if the problem is known, NGT generates possible solutions. These 
solutions are synthesised in the evaluation phase during which the group screens 
and selects the 'strategic elements of a problem or component elements of 
alternative solutions' (Delbecq et al. 1975:9). 

Claim 2. Balanced participation is fostered by the procedure of silent and 
independent generation of ideas and round-robin listing, a process which 
according to Delbecq et al. (1975:33) 'facilitates the self-disclosure of ideas 
even by less secure members 1 who may hesitate to speak in conventional group 
discussions. Moreover, 'the round-robin procedure of writing items on a 
blackboard or flip-chart reduces arguments over semantics, increases retention 
of ideas, and decreases redundancy of discussion'. This process is the key to 
the method. It aims to prevent the problem which can happen in the 
conventional interacting group discussion, of a few people dominating the 
discussion. 

Claim 3. The voting procedure and rank ordering of items enables group 
consensus to be established relatively easily. 

The silent generation of ideas, a method which effectively ensures that each 
person will be heard, is claimed to provide adequate time for thinking and 
reflection; a process encouraged by the presence of others similarly engaged. 
The method is said to foster decision making by the avoidance of interruptions; 
the avoidance of undue focusing on a particular idea or train of thought; 
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avoidance of competition, conformity and status pressures, that is, the method 
claims to reduce the influence on a group by dominant members possessing 
personal magnetism, high social status or hierarchical position-power (Delbecq 
et al. 1975:9). 

The round-robin listing of ideas permits equal partir ation in ;he 
presentation of ideas; an increase in problem-centredncss; t separation of 
ideas from personalities; an increase in the ability to deal with a larger 
number of ideas; tolerance of conflicting ideas; encouragement of the 
stimulation of other group members by the ideas emerging; pnd the provision of 
a written record (Delbecq et al. 1875:47). 

The method claims to improve the task-focussed communication structure of 
groups by remaining problem-centred, rather than people centred. Furthermore, 
the method avoids choosing between ideas prematurely since ranking of ideas is 
delayed until all ideas are listed. Another claim is that the method fosters 
'hitch-hiking' (latching onto and building upon the ideas of others), leading to 
greater quality of views expressed. This claim is supported by Roe and 
McDonald (1983:33) who state that a well handled NGT 'can provide 
considerable . insight into the views of any group of staff or students...'. The 
Nominal Group Technique can be used as a bridge between 'supervisors and 
subordinates, and between teachers and administrators' and is 'a means of 
identifying the problems perceived to exist between such groups' (O'Neil, 
1981:48). So NGT can be used to detect areas of tension affecting planning or 
implementation. 

Despite its authoritarian character (power strongly centred in the hands of 
the leader, leader-member dialogue permitted not member-member dialogue), 
NGT was designed to overcome a » .lem which can arise in groups: the effort 
required by the facilitator in handling emotional factors (tension, conflict, 
dominance) rather than task-oriented factors. 

Delbecq et al. (1975) also present the view that interacting groups often 
display a reluctance to attack the task by such practices as diversionary 
questions, personal statements or ideologies expressed regarding the task. 
Ideologies embody, among other things, sets of beliefs which obscure or hide 
the real fscts (Billig, 1979). 

According to Delbecq et al. (1975:34) NGT includes a number of 
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characteristics which facilitate decision-making performance: 

1. Low variability among groups in member and leader 
behaviour leads to consistency in decision making. 

2. A balanced concern for socio-ernotional group maintenance 
roles and performance of task-instrumental roles offers both social 
reinforcement and task accomplishment reward to group members. 

3. The silent independent generation of ideas, followed by 
further thought and listening during the round-robin procedure 
results in high quality of ideas. 

4. Search behaviour is proactive (as opposed to 'reactive', see 
Note 1, p.88), characterised by extended periods in generating and 
clarifying alternative dimensions of the problem, tendencies for 
high task-centred group effort, and the generation of new social 
and task-related knowledge. 

5. The structured process forces equality of participation among 
members in generating information on the problem. 

6. NGT meetings tend to conclude with a perceived sense of 
closure, accomplishment, and interest in further phases of problem 
solving. 

4. Conducting an NGT workshop 

The main steps, their objectives and the guidelines for the facilitator role, 
based largely on Delbecq et al. (1975), are set out in Table 3.1 and then 
described in detail. 
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TABLE 3.1 

FACILITATOR GUIDE TO CONDUCTING AN NGT WORKSHOP 



Steps/objectives 



Facilitator role 



THE NGT WORKSHOP 

Step 1: Silent generation of 
ideas in writing. Objectives: 
to map the group's thinking 
and enable equal participation. 



Step 2: Round-robin listing 
of ideas. 



Have the NGT question ir, 
writing. Explain the task is 
to write responses silently 
and independently, in brief 
phrases or statements. Do not 
discuss the question or allow 
interruptions except to 
explain the process. 
Illustrate the level of 
abstraction and scope with an 
example which does not distort 
(lead) group responses. Remain 
silent yourself while group 
works. Do not walk around the 
group. Quieten any disruptions 
by addressing comments to the 
group as a whole. 

Explain the step is to map the 
group's thinking. Ideas should 
be presented in brief words or 
phrases and will be taken one 
at a time from each member. 
Don't alter ideas when 
recording them. Write quickly. 
Make the list visible to all. 
Eliminate duplicate items only 
if the group agrees. No 
discussion but encourage 
'hitch-hiking 1 and adding of 
new ideas not previously 
recorded by members. (The 
'no discussion' rule is meant 
to avoid argumentation). 

Vary the rule if the ideas are 
flowing, but reaffirm the 
principle of no criticism or 
evaluation at this stage. 
Allow individuals to 'pass' 
but to re-enter at any time. 
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Sstpe/ofejectivtft 



Facilitator r« ■< 



Step 3: Serial discussion of Define the (ask as 

♦<**•* for clarification, 'clarification'. Minimise 

Objectives: iu r,« 0 re thai •rfumenuilon but do not be 

each item is umjor.t.xwj heavy handed. Explain that 

clearly and that ail it* final judgment will be 
are covered, expressed by voting. 

Allow speakers to state the 
logic or facts behind any 
Item. Encourage addition of 
new ideas sparked from the 
ideas of others. Avoid forcing 
any member to be the sole 
person responsible for 
clarifying his/her Item. 

NOTE; If items are separate, proceed to next step. If Items 
overlap a clustering or 'collating' step should be added 
at this stage In order to simplify and organise the listed 
Items. Do not merge Items without agreement of the 



Explain the voting 
procedure, or negotiate 
(rank ordering or rating 
scale use) with group. 
All items to be ranked. 
Ranking is done privately 
on Index cards then collected, 
shuffled (to protect anonymity) 
ai»d the results recorded 
beside the Items on the 
flip-chart. 



Step 4: Preliminary vote on 
item importance. Objectives: 
to enable Individuals to make 
Independent Judgements; to 
express these by rank ordering 
or rating items; to use the 
arithmetical mean as the group's 
decision to present the results 
for further discussion and 
re-voting. 
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Steps/objectives 



Facilitator role 



Step 5: Discussion of 
preliminary vote. 



Define task as clarification, 
not social pressure. Keep 
discussion brief so as not to 
distort perception of Items 
not discussed. Avoid being 
heavy handed. Some members 
whose ideas were not highly 
ranked may feel that their 
importance was not 
appreciated. Caution 
members to think carefully 
about any changes they make in 
their voting. 



Step 6: Final vote. 
(Repeat Step 4). 



Planning and conduct of NGT 
Writing the NGT question 

Crucial to the conduct of a successful NJT exercise is the wording of the 
NGT question which will focus the group's work. Delbecq et al. (1975:76) 
note that it is virtually impossible to frame an NGT question unless the 
objective (note the use of the singular 'objective 1 ) of the meeting is clear. 
They advise the following steps. 

•I. Staff discussion of the objective of the NGT meeting. 

2. Staff illustration of the type of items sought 

In terms of (a) level of abstraction, (b) depth versus 
breadth. 

3. Staff development of alternative questions. 

4. Pilot testing alternative questions with a sample group. 9 

According to Hegarty (1977:33): 

The task posed by the NGT question must be phrased in such a 
way that participants can make direct use of their own knowledge 
and experience ... it is stressed that all types of problems are 
wanted - feelings and attitudes as well as educational, scientific, 
administrative and technical difficulties 
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A crucial concern with the NGT question is what level of information is 
needed: Information at a general level (i.e. course aims) or at a detailed level 
(I.e. tasks). 



When the composition of the group includes people whose technical expertise 
does not cover the whole scope of the area under investigation, the group can 
be asked to list critical incidents or examples or descriptions of behaviour for 
the areas known to them rather than list categories or general problem labels 
as these can be extracted later. (See the discussion of the Critical Incident 
Technique in Section 3.3). These can then be analysed to find 'core 1 
competencies and hence course aims or subject aims. 

Examples of 'broad 1 , 'narrow' and 'personal' NGT questions include: 'What are 
the weaknesses associated with this course' (BROAD); 'In the light of potential 
needs of the late 1980s, what do you personally think are the important 
ingredients In the revised B.Ed, course?' (BROAD); 'List problems which you 
have encountered in connection with the goals, objectives, learning program and 
outcomes of the microbiology course for final year science students at this 
university' (BROAD) *What are the really good aspects of this course unit - the 
sort of things that would make you recommend it to other people?' (BROAD) 
'What aspects of this course unit do you think should be changed?' (SPECIFIC). 
•What difficulties have you experienced in meeting assignment deadlines?' 
(NARROW); *What insights have you gained as a consequence of participating in 
this learning group?' (PERSONAL). 

Group size 

NGT is suitable for groups of up to nine or ten people. A small meeting 
room is required with seating arranged around a table with a flip-chart at one 
end, or In a *U f shape. The method can be used with a number of small 
groups working simultaneously. 

You will need: 

• flip chart 

• roll of masking tape 

. pack of cards approximately 13 X 8 cm 

• felt pens 

. paper and pencil for each participant 
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Welcoming statement: 

• give a warm and cordial welcome 

• give a sense of importance to the task at hand 

• state the importance of each members contribution 

. give an indication of the use or purpose of the meeting's 
output. 

The NGT workshop 

Step 1: Silent generation of ideas in writing 

In this step members are presented with the nominal question in written 
form. The aim is to generate, in Step 2, a 'master list 1 of ideas. The leader 
reads the question aloud, then asks the group to write ideas in brief phrases or 
statements while working silently and independently for about 5-15 minutes 
depending on the task (longer in special circumstances). The leader should 
resist f non-process clarifications 1 that is should not discuss or explain the 
question but only the process of addressing it, i.e. by silent generation of 
ideas. The rule of not discussing the question is intended to avoid giving 
verbal or non-verbal cues as to how it should be answered. Members who 
disturb the silence of the group should be sanctioned (Delbecq n al. 1975:45). 

Step 2: Round-robin recording of ideas 

The ideas of the group are taken one at a time f"om each person and 
recorded on the flip-chart visible to the entire group. No editing of material 
or evaluative comments are allowed. The leader guidelines for this step are: 

f l. Clear verbal statement of the step: 

a) the objective is to map the group's thinking. 

b) ideas should be presented in brief words or phrases. 

c) ideas will be taken serially. 

2. Effective mechanical recording. 

3. Direct sanction of inappropriate group behaviour. 1 
(Delbecq et al. 1975:49). 

Examples. 'Mr Smith, would you give me one idea from your list 1 . If group 
member states that he/she has no more ideas, the leader can move on to the 
next person but should say something encouraging like: f ...feel free to 
contribute another idea if you want to 1 . 

Recording guidelines. Record ideas as rapidly as possible, in the words used 
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by the group members (very important). Provide assistance in abbreviating only 
in special situations. Make the entire list visible to the group by placing 
completed sheets on the wall. In cases where a person is having difficulty 
presenting an idea in clear, simple terms, the leader might ask the person to 
think the idea over in order to turn it into a few words or a short phrase. 
The reasons for recording ideas in the contributor's exact words are that it 
increases the perception of equality and values the contribution as important, 
gives greater personal involvement in the task, and eliminates any possibility 
that the leader is manipulating the group (by altering or deleting words). 
Sometimes during this phase group members will spontaneously begin to discuss 
the ideas or note similarities with ideas already stated. The leader should 
resist discussion until all ideas are listed, saying 'let's wait until we have heard 
all the ideas'. If a group member feels that his/her idea is different from one 
already stated, the leader can place the responsibility back on the group 
member, e.g. 'If you feel your idea is slightly different, let's put it up on the 
chart'. This stage may spark additional ideas by allowing members 'to 
periodically 'interact' with the group, and, more important, to 'hitch-hike' on 
the ideas of others' (O'Neil, 1981:49). 

Step 3: Serial discussion for clarification 

Each idea is discussed in turn. The benefits of this are: 

'(1) avoidance of focusing unduly on any particular idea or 
subset of ideas, (2) Opportunity for clarification and elimination of 
misunderstanding, (3) Opportunity to provide the logic behind 
arguments and disagreements, (4) Recording of differences of 
opinion without undue argumentation'. (Delbecq et al. 1975:52) 

The central objective in the discussion stage is to 'clarify, not win 
arguments' and 'to minimise influence based on verbal prominence or status' 
(Delbecq et al. 1975:52). The purpose of clarification is so that the other 
members can understand the meaning of the brief words or phrase. It can 
include explanation of the logic or analysis behind the item and the relative 
importance placed by the contributor on the item. 

During this stage the leader should 'pace' the discussion, that is, not allow 
discussion to: unduly focus on any particular item, or to degenerate into 
argumentation. However, differing points of view should be aired but the 
leader then move the discussion onto the next item (Delbecq et al. 1975:53). 
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Note again that the purpose is to disclose thinking and analysis, not to resolve 
differences of opinion. T*' 1 authors argue that differences of opinion will be 
'accurately recorded in the voting procedure 1 (1975:53), 



The role of the leader in this step is stated by Delbecq et al. (1975) as. 

1. 'Verbally define the purpose as clarification, 

2. Pace the group to avoid undue argument or the neglect 
of certain items at the expense of others 1 . 

When an argument develops the leader can say: 'I think we understand both 
points of view at this point. Perhaps, however, we should move onto the next 
item in the interests of time 1 (Delbecq et al. 1975:54). The authors note that 
•groups generally pace themselves if the leader clearly indicates the available 
time for the step! 

Step 4: Preliminary vote on item importance 

Delbecq et al. (1975:56-61 recommend the following rank ordering voting 
procedure. 

'1. Ask participants to select from the flip chart sheets the 
five most important items and to record the number of each item 
on the cards provided (in say, the upper left hand corner), 
then to write the identifying words or phrases on the card (in 
the centre). If the list of items is long you may ask for up to 
eight items to be selected in order of importance. 

2. Ask participants not to rank order the actual cards at this 
stage. 

3. When participants are ready, ask them to spread out their 
cards and select the most important one first (or the least 
important one) and to write the number of the rank which the 
person wishes to give to the item in one corner (e.g. the bottom 
right hand corner) and to proceed until the ranking process is 
completed. 1 

In some cases it is helpful to give the group criteria for the ranking, e.g. 
'what five items are central in preparing a student for work in ...?'; or, 'which 
five items tell us what theory knowledge (or what practical knowledge) a 
student must have'; 'what five items tell us most about the likely changes in 
the occupation in ...' (specify time frame) (See Note 2). If criteria are used, 
these should fit within the logic or scope of the NGT question. Other t*sers of 



ERIC 




NGT have adopted the practice of asking the group to rank the ten top items 
(O'Neil, 1981). 

Below is an example of an index card illustrating the rank order voting 
procedure* 



5 (Item number from flip chart) 
(Summary of actual item) 
Member's rank X 

Step 5: Discussion of preliminary vote 
The aim ot the discussion is to: 

1. Examine inconsistent voting patterns. 

2. Provide opportunity to re-discuss items which are 
perceived as receiving too mary or too few votes. 
(Delbecq et al. 1975:62) 

The leader defines the task as •clarification, not social pressure 1 and ensures 
that discussion is brief (Delbecq et al. 1975:63). The leader tries to ensure 
that the voting is not influenced by errors due to misinformation, 
misunderstanding, or unequal information (hence the importance on the rule of 
seeking clarification). However, Delbecq et al. (1975:63) note that discussion 
can decrease accuracy. They suggest the leader gives a statement at the 
beginning and the end, along the following lines: 



At the beginning 

It may be worthwhile to briefly examine the voting 
pattern ... to see if there are any inconsistencies, 
surprises, or differences members wish to comment on. 
The purpose of this discussion is not to pressure any 
member to change his or her vote ... if we gain additional 
clarification some members may wish to modify their 
original vote. 

At the end 



... the purpose of this discussion has not been to get 
you to change your original vote ... however, if you 
honestly have a new perspective as a result of the 
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discussion, you should change your vote. 

Another rating method (O'Neil, 1981) is to use a 5-point Likert-type scale 
for each item generated in the workshop, for example, 



STRONGLY DISAGREE 

AGREE 1 2 3 4 5~ 

The ratings are summed and the average calculated (average = sum of the 
scores, divided by the number of cases). Rating scales are useful if greater 
'judgemental accuracy' is needed but 'depends on the degree of information 
possessed by the group' (Delbecq et al. 1975:63). If a long list of items is 
generated by the group, individuals can rank the items in order of importance 
then write these in a column beside each item on the flip chart sheets. By 
scanning the results an approximation of the importance of items can be 
obtained and then these items extracted. 

Step 6: Final vote 

The final vote: 

1. Determines the outcome of the meeting. 

2. Provides a sense of closure and accomplishment. 

3. Documents the group judgment. (Delbecq et al. 1975:63). 

The procedure for rank ordering the final vote depends on the degree of 
refinement sought. In some cases where the topic is general, the procedure in 
Step 4 (preliminary vote) can be repeated in which priorities are simply 
rearranged. If fine detail is sought, Delbecq et ai. (1975:64-5) suggest either 
of the following methods shown in Tables 3.2 and 3.3: 
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TABLE 3.2 

NGT RATING FORM FOR FINAL VOTING: VERSION 1 



No. from 

flip chart Item description Most important 



100 
90 
80 
70 
60 
50 
40 
30 
20 
10 
0 



Least important 



Instructions 

1. Choose the five most important items from the flip chart and 
list them in rank order above. 

2. Identify the item by using the number and description from the 
flip chart. 

3. Draw a line from the item to the scale (0-100) at the right, 
indicating the relative importance of each item. 



oi 
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TABLE 3.3 

NGT RATING FORM FOR FINAL VOTING: VERSION 2 



No. from Relative importance 

flip chart Item description Not important Very important 



Instructions 

1. Choose the five most important items from the flip chart, and 
list them in rank order above. 

2. Identify the items by using the number and description from 
the flip chart. 

3. Rate each item in terms of its importance on the 0-10 scale, 
with 0 being unimportant, and 10 being very important. 



Analysis of data 

The data from an NGT workshop may need amplification since NGT can 
generate items which require further research. 

Suitability and value of the method in TAFE curriculum development 

According to O'Neil (1981:45) the method is suitable for needs assessment 
because it allows assessment of "consumer (student/teacher) rather then 
producer (lecturer/project leader) interests 1 . Curriculum research applications 
of NGT have included curriculum planning, course content evaluation, 
determination of learning outcomes and needs of students for a professionally 
relevant course (O'Neil, 1981:45). 
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The Nominal Group Technique is appropriate, according to Delbecq et al. 
(1975:81) when it is required to: Identify the elements of a problem situation, 
identify elements of a solution program and to establish priorities. NGT works 
whenever the decision task is complex and 'calls Vor the pooling or aggregation 
of individual judgments' and is not so appropriate 'when the aim of the group 
meeting is information exchange and co-ordination 1 . In the latter case (sharing 
and exchange of information) the conventional group discussion is better (Ford 
and Nemiroff, 1975:182). 

Modifications 

Using NGT with larger groups 

Delbecq et al. (1975:70-72) suggest dividing the large group into groups of 
5-7 persons and after Step 4 (preliminary vote) the leaders of each group meet 
to prepare a master list. In this process, each leader explains the top five 
items identified by his/her group, then each leader independently combines 
duplicate items and any disagreements are discussed. Next, a carefully worded 
statement is prepared which will convey the accurate meaning for both the 
combined and individual items. This, together with the original rankings, is 
presented to the whole group which: 

• Discusses and clarifies the master list. (Clarification 

is essential as some statements will have been altered.) 
. Discusses the preliminary vote and the relative importance 
of items. 

• Makes any additions. 
. Votes. 

In an application of NGT to a university level microbiology course, four groups 
were used: Representatives of students, teachers, a group with an understanding 
of the context (i.e. who understood the relationship between various units in 
the course and the relationship of microbiology to other disciplines), and a 
group of subject matter specialists and curriculum planners (Hegarty, 1977:39). 
This latter group included a significant number of people who would be 
involved in curriculum design. 

NGT in exploratory research 

Exploratory research or pilot research involves the investigation of complex 
problems whose qualitative and quantitative dimensions are unknown (Delbecq et 
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al. 1975:109). They contend that NGT can be used to 



'1. Identify and enrich the researcher's understanding of n 
problem by providing judgmental item identification, 
refined by priority ranking and voting, 

2. Arrive at a set of hypotheses concerning the meaning and 
effects of determinate aspects of the problem areas. 

3. Focus attention on the major areas of inquiry, defined by 
target groups in their own jargon, which may be pursued in 
greater detail later by means of interview or questionnaire 
instruments 1 (Delbecq et al. 1975:112). 

However, the following methodological difficulties should be noted. 



. The selection of target groups for problem exploration. 
. The s n ecification of the question to be used in NGT 

meetings for gathering data from target groups. 
. The transformation of this raw data into standardised 

measurement instruments. 

U) exploratory research applications the formulation of the NGT question should 
jf\iT in mind the following question: 'What insightful or crucial information does 
each target group possess that would help us to understand the dimensions of 
the problem' (Delbecq et al. 1975:114). They contend that NGT can be used 
with a broad question in order to 'elicit varied responses'. Subsequent meetings 
or NGT programs can be used to refine the question. 

O'Neil (1983) used NGT data in an evaluation questionnaire by the simple 
procedure of turning the data into questionnaire items using a 5-point scale: 1, 
Strongly disagree. 2, Disagree. 3, Neither disagree nor agree. 4, Agree. 5. 
Strongly agree. 

Costs and time 

Depending on the problem, the number of simultaneous groups working and 
the number of items generated, NGT can take from two hours to two or three 
days, even more if conducted on a number of target groups. In a detailed 
curriculum development exercise, the process may begin in the conventional 
form in order to generate the dimensions of the problem. These dimensions 
codd then be addressed by other methods such as DACUM-type methods aimed 
ai stating the knowledge and skills required by the competencies identified by 
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the NGT process. Or, NGT could be repeated using new NGT questions on 
aspects of the course, e.g. what should be the aims or the content for 
particular subjects or electives. 

O'Neil and Jackson (1983:lo0-l) give the following guidelines for the time 
taken in some of the stages when they reviewed a B.Ed. course with the 
teachers. 



Step 1: Silent generation of ideas - 5-20 minutes. 

Step 2: Round-robin listing - 45 minutes. 

Step 3: Clarification - 5-10 minutes. 

Step 4. Merger of items - 20-30 minutes. 

Step 5: Preliminary voting - 10-15 minutes. 

The above times should be taken as the minimum needed. 
Conclusion 

The Nominal Group Technique has a number of strong advocates. Indeed, 
one TAFE curriculum developer commented that it was probably the single 
most useful technique in his repertoire. The Nominal Group Technique appears 
to be suitable in situations where a group agrees that a problem needs to be 
solved, are uncertain how to solve it, but are willing to attempt the task. 
Willingness to accept the relatively rigid structure of the process, especially 
the rule of 'no discussion except for clarification' appears to be an important 
element in successful applications of NGT. Where strong task-nchievement 
motivation is present in the selected group, the method should be effective. In 
situations where the problem task contains a highly charged emotional layer, 
the method should be approached with caution or should be presented as an 
exploration of the task dimensions of the problem, to be followed at a later 
time, by careful study and discussion of the results and, in particular, close 
attention to any strongly held views that were discounted by the consensus 
voting. 

The problem with all consensus measures, including statistical analysis, is 
that items which may be important but which receive low scores become 
ignored. A review of ranking scores might, therefore, ask what was overlooked 
- were any important insights or solutions or alternatives lost in the ranking 
such that cracks or flaws which these reveal are followed up. The importance 
of following up minority views carmot^be ^vep-empbasi^ Yn., n, ny have in 
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an NGT group one person who is actually providing very deep insights into the 
matter under examination; insights which may be lost in the consensus ranking 
procedure! 

NGT, by requiring an initial, silent generation of ideas, requires participants 
to focus on what they personally feel is salient in defining the problem or 
solving it. ihat is, NGT assumes that the force of good ideas will carry the 
day through consensus as opposed to weight of argument or the position 
(authority) of the proposer. The central assumption is that conflict is almost 
certain to exist within the group and can be addressed by providing a means, 
the silent generation of ideas and round-robin listing of member's contributions 
for providing data for the group to evaluate. 

Ford and Nemiroff (1975:182) concluded that while NGT appears to be 
especially functional for fact-finding or idea-generating tasks, the discussion 
group seems superior for sharing and evaluating information. 

The idea of combining aspects of NGT with group discussion methods seems 
promising. A simple modification of the NGT procedure, which would offend 
the NGT purists, would be to relax the rule of 'discussion for clarification only 1 
and, with the group's agreement, set aside a period of 'free discussion' before 
resuming the NGT procedure (if at all). 

In problem identification applications, the problems identified by NGT can 
then be the basis for a Force Field Analysis to identify positive action steps. 
(See Section 3.4). In curriculum development applications the list of duties 
identified, e.g. by a DACUM procedure can be sorted by the NGT voting 
procedure into levels, e.g. higher (technician) level or operator level and work 
can then procee a the level being addressed. In instances where the group 
lacks comprehensi ; /owledge of the occupation being analysed, some further 
research will be nt 2 ded. The initial NGT group should be made as 
representative as possible. 

Some things to bear in mind when considering NGT are: 

. Select participants who will contribute and who have a 
strong motivation to address the task, that is, do not 
have the type of 'hidden agendas' which will lead to 
attempts to obstruct the method. 
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• Do your homework. Know the area in which you are 
conducting an NGT, so as to avoid being deceived if the 
NGT does not provide a full picture. 

• Identify group leaders who will try to harness the 
energies of others. 

. Keep your objectives in sight. Use those elements of 
the process which suit your purposes and discard those 
which do not. 

. Involve those who will be affected by the results. 

Finally, the method can cause frustration if applied to the wrong type of 
problem or is applied insensitively or in an excessively rule-guided way. The 
method reflects American cutlural values on achievement and efficiency. 
Caution is indicated when considering the method for use on groups whose 
cultural values are not American. Be prepared to abandon the method if things 
go wrong on the day by switching to another group discussion process, or 
possibly, a Search Conference format. 

Notes: 

1. The term 'proactive 1 is attributed to NGT because each 
member is 'required to write and/or articulate his/her 
views without the opportunity for other group members 
to react or evaluate until all ideas are presented' 
(Delbecq et al. 1975:23). 

2. This type of question may better suit a Delphi process. 
3,2 THE DACUM METHOD AND DERIVATIVES 

Background to the DACUM method (original or 'classical' form) 

The DACUM method, like the Skills/Knowledge/Attitudes (SKA) method, has 
its origins in the 'systems approach' to training. It is concerned with the 
'what' of curriculum as opposed to the 'how' of curriculum. In this way, it is a 
'task analysis' technique used for the systematic development of a curriculum. 
It is concerned with the identification of skills rather than concepts or 
understanding. 

DACUM (short for 'developing a curriculum') is an approach to the 
development of curricula combined with an evaluation process for occupational 
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training programs. It was initially developed conjointly between the Canadian 
Department of Manpower and Immigration and the General Learning 
Corporation of New York, This initial development was intended to produce a 
curriculum guide to enhance trainee development in the program and in 
planning for goal achievement. Following this early effort, an 'experimental 1 
DACUM was developed in Canada and was introduced *~> the New Start 
Corporations in Canada in 1968. It was shortly afterwards adopted by Nova 
Scotia New Start Incorporated which was seeking a new approach to curriculum 
development to enable it to respond quickly to the training needs of 
disadvantaged adults. 

Various forms of modified DACUM have been used in Australian TAFE since 
the late 1970s. DACUM seems particularly suitable for use in TAFE curriculum 
research - evidenced by the number of TAFE curriculum practitioners who have 
used it in some form. As well, the emergence of DACUM from the systems 
approach and its concern with generating behavioural learning outcomes 
(Sinnett, 1974) means that it is consistent with the theoretical frameworks 
adopted by a significant number of curriculum practitioners in TAFE. 

The writers 1 interest in the DACUM method is therefore stimulated by: 

. its fairly widespread current use in a modified form in 
several States in Australia; 

. its apparent consistency with the approaches to curriculum 
development adopted by some TAFE curriculum practitioners; 

. its claimed suitability for the generation of programs 
which provide adult learners with an opportunity to learn 
in ways which suit their individual characteristics and 
backgrounds (i.e. is androgogical), rather than being an 
instructional model with pedagogical underpinnings 
(Adams, 1975); 

. its apparent speed of execution, compared to conventional 
quantitative methods. 

The DACUM rationale 

O'Hanlon (1973:74) identifies three approaches to curriculum development and 
asserts that there are models which are illustrative of each approach. 
O'Hanlon identifies the 'management' approach, the 'systematic' approach and 
the 'open-access' approach. Management models are characterised by decision 
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making control being retained at the top of the hierarchy. Systematic models 
are concerned with specifying the goals to be attained and then making 
decisions according to educational philosophy in order to achieve these goals. 
Open-access models are based upon the humanist tradition and are 
characterised by decentralisation of decision making. None of these models 
stress the importance of researching the conceptual knowledge needed to 
perform complex work, perhaps because vocational work is seen as 'manual 1 
work, not 'mental 1 work. 

In reference to O'Hanlon's three models, Sinnett (!974) classifies DACUM as 
a 'systematic' model. As such, it first establishes the aim or purpose of the 
curriculum. Decisions regarding the development of the curriculum are 
governed only by this purpose and hence persons other than management should 
also be involved. A task analysis is performed carefully and each stage is 
evaluated in terms of whether it is furthering the original purpose. Resources 
are allocated only on the grounds that they help achieve the stated goals. 

An overview of the DACUM method 

The DACUM process as it is being reviewed here, was seen by its original 
developers as just a part of a whole - a whole instructional system which was 
specifically designed for adults in a vocational training situation. The process 
and product, i.e. the DACUM group workshop and the charts of skill 
definitions, have been extracted from the more thoroughgoing original DACUM 
approach of Sinnett (1974) and Adams (1975) by the writers since these are the 
parts that are pertinent to our research. Notwithstanding this extraction, the 
presentation of DACUM that follows represents the classical or original form of 
DACUM. A number of DACUM derivative methods have been used by 
practitioners in Australian TAFE - these are described later in this section. 

In the DACUM process, a co-ordinator or workshop group facilitator works 
with carefully selected individuals who possess a range of experiences in the 
occupational group which is under focus. By guiding group discussion and 
gaining information from the group members, the facilitator builds up a skill 
profile of the occupational group being studied. 

The facilitator prints short definitions on file cards which are affixed to a 
large blank wall in a structured pattern. The definitions are short, begin with 
a verb, and taken together should cover the entire range of activity within the 
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occupational group being studied. This analysis proceeds from a fairly high 
level (or broad range) or activities to increased levels of specificity. The 
degree of specificity is carefully monitored by the facilitator. 

The cards can be arranged or sequenced according to an appropriate 
rationale such as level of difficulty, subject/module taxonomy, or the nature of 
the job. By numbering the completed arrangement of cards the information is 
transfer ed and transcribed onto sheets printed with blank squares. These 
sheets can then be used as the basic curriculum development plan and are 
referred to as DACUM charts. (See Appendix D.) 

The facilitator 

The role of the facilitator in conducting curriculum research workshops has 
been explored in some detail by the writers in their report entitled: The 
Facilitation of Curriculum Research Workshops in TAFE (1986). The reader 
interested in studying the facilitation role in detail is advised to consult that 
report. Some brief comments on facilitation are, however, noted here. 

The facilitator is a key person to the DACUM exercise. The art of 
facilitation is best learnt by doing - initially as a co-facilitator with an 
expert. A DACUM facilitator should have a fairly broad academic background 
and possess good interpersonal skills. He/she should have experience in group 
handling and should have a facility with words. Experience with curriculum 
design and analysis techniques is also important, although a knowledge of the 
subject area under study is not necessary - indeed some DACUM practitioners 
consider such knowledge detracts from the facilitator's performance. 

Selecting and orienting the workshop group 

Avoid selection of participants for 'political' reasons - such people will 
probably have vested interests and may make the facilitator's job unnecessarily 
difficult. The most useful people to involve in the DACUM workshop are those 
who are actually performing, or supervising the job under study, or who are 
directly involved with the endeavour in some way. The following guidelines 
may be useful: 

. Workshop participants should represent a 
cross-section of the business, industry or community 
grouping. 
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. Geographical representation should be attended to. 

. Different skill strata within the target group, e.g. 
servicing, manufacturing, distributing, developing, 
selling, should be represented. 

. Workshop size should not be less than six or more 
than about 14 - some practitioners contend that about 
ten is an ideal size. 

. Workshop participants need to be able to attend for 
the full period of the workshop (generally two days) 
- some form of remuneration or reimbursement of expenses 
(perhaps even a special dinner) should be arranged. 

. The composition of the group should be balanced. If for 
some unavoidable reason bias occurs in the 
representativeness of the group, then a series of 
complementary DACUM workshops could he held and the 
individual outcomes (charts) synthesised. 

. Account should be taken of the participants' ability to 
verbalise, keep abreast of their own field and to make 
predictions and projections about future developments 
within the field. 

. Contact potential workshop participants individually prior 
to the workshop and provide a rationale and motivation 
for the DACUM workshop. 

. Public relations and managerial personnel who are not 
directly familiar with the job as it is being practised 
now should not be included. Teachers/lecturers have 
generally developed biases about how a program should be 
taught - and hence should not be included according 
to Finch and Crunkilton, (1979:115). Intending 
teachers/lecturers of the program being studied should 
be encouraged in small numbers, to sit in on the 
workshop as observers only. 

. A pre-DACUM workshop session should be conducted - most 
conveniently at the beginning of the 2-day workshop. 
This should comprise a general description of the DACUM 
method and its rationale - some slides or a video or 
film are a useful adjunct. 

Workshop facilities 

Figure 3.1 illustrates an arrangement of the workshop room suitable for 
conducting of a DACUM session. The facilities should include: 



. A room with at least ten metres of plain unbroken 
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wall space for affixing the cards. 

. Comfortable chairs, preferably swivel type with arms. 

. Enough low tables or desks to line up along the complete 
length of the wall. This row of tables serves to separate 
the seated participants from the wall - it also serves as 
a place for charts, coffee and ashtrays. This 
arrangement focuses attention on the wall, but 
discourages any participant from leaping up and 
rearranging cards without the facilitator gaining 
consensus from the group about such a rearrangement. 

. Tea/coffee should be available throughout the 
workshop. The workshop group should be discouraged from 
breaking for tea/coffee or lunch - it is best if 
^sandwiches' (or the like) are pre-arranged and served 
in the room. Breaks in the process should be avoided lest 
the impetus be lost. 

. A blackboard or large sheets of butcher's paper for 
outlining the occupational areas under study. 

. A supply of felt pens. 

. A large printed sign stating 'the individual/student 
must be able to ...' This should be fixed on the wall, 
fairly high up. 

. A supply of file cards preferably of two different colours 
and sizes - e.g. 12 cm X 15 cm pink and 10 cm X 12 cm 
white. 

. A supply of 'Blue-tack' or some other adhesive to stick 
the cards to the wall. 

. Three 'metre' sticks with prepared spots of 'Blue-tack' 
already placed on. This is handy for placing cards on the 
wall quickly. 

. Sample verb (or action word) lists taken from other 
charts, curriculum documents or lists of instructional 
objectives. These can be used if it appears that 
participants are having difficulty verbalising 
occupational skills. 

. Some form of participator identification sheet listing each 
participant. 

. The workshop location is best relocated away from the 
college premises to avoid such interruptions as college 
tours, or unwanted visitors. 

A suggested sequence of DACUM steps is listed below: 

. General introduction and orientation. 
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. Review of the occupational area. 

• Identifying the general areas of competence, 

• Identifying the specific skills for the first of the 
general areas of competence - i.e. completing the first 
group of duties, 

• Identifying the specific skills for each of the 
remaining general areas of competence, 

• Reviewing and refining the definitions of duties. 

• Sequencing the skills. 

. Establishing levels of competence for each skill in 
terms of difficulty, frequency and general importance 
to the occupational area. 

. Final structuring of the chart and recording 
conclusions from the workshop session. 

Conducting a DACUM session 

The following description of the conduct of a DACUM session is presented in 
the sequence of nine steps listed immediately above. 

General introduction and orientation 

The length and concentration of this session will depend upon the degree to 
which the workshop organiser has been able to meet and talk to the individual 
participants during the planning phase for the DACUM workshop. The aspects 
that should be covered either before or at the beginning of the session are: 



. an overview of the DACUM process so that participants 
understand the importance of identifying behaviours or 
activities rather than knowledge or concepts (hence the 
sign on the wall); 

. provide a short resume of own (facilitator's/organisers) 
background; 

. provide background on the need for the program (course) 
under review to be reviewed - e.g. by students 1 , teachers' 
or employers' complaints about existing course or 
technological changes; 

. provide some background information on the reviewing 
and approving mechanisms that apply to the curriculum 
being examined; 
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. using an existing DACUM chart, explain how the profile is 
set up, how the learning objectives will come from the 
definition on the chart and how the course will be 
organised around it; 

. emphasise the importance of the participants 1 
contributions in developing the DACUM chart. 

Review of the occupational area 

It is important that participants quickly focus on the representative person 
they are about to analyse. They need to have a mental image of a real or 
proposed set of competencies possessed by some individual involved in the 
occupational area. This can be done by, for example, dividing the work area 
concerned into a number of main categories such as professional, technical, 
tradesperson, or operator. Encourage participants to provide the names or 
designations of job titles within the field. The facilitator should aim to fit 
these together on the chalkboard so that the whole work area is clarified. The 
typical mobility within the work area should be established and the single or 
group of occupational categories which rhey are going to analyse must be made 
as specific as possible. 

Identifying the general areas of competence 

Movement into this stage should be achieved as quickly as possible. For 
this activity use the larger of the two types of file cards that have been 
prepared. The facilitator asks participants to name 'duties' or broad areas of 
work which occupy a significant part of the time of the practitioner on the 
job. These duties or general areas of competence should not be specific tasks 
- they should be able to be broken down into specific tasks later (if not, then 
they do not constitute duties). 

Record these duties on the cards (recording can best be done by a 
co-facilitator). The description of the duty should include a verb or imperative 
and be of the order of six words in length. Place cards as quickly as possible 
on the large wall in a vertical column on the right hand side. Regrouping 
cards and changing the wording on cards can be done any time and as often as 
necessary throughout the workshop. 

Group the duties or general areas of competence into like groupings under 
headings such as supervisory, administrative, use of tools, analytical problem 
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solving, or the like. 



Analysing the first group of duties 

Choose any of the general areas of competence that appears to be fairly 
straightforward to work on first - e.g. something manipulative rather than 
analytical. 

Since DACUM focuses on what work is done in an occupation, the guiding 
principle for the facilitator at this stage is to insist that participants make 
their contributions in terms of 'The individual/student must be able to and 
not to accept contributions in terms of 'the student should know ...' or 'the 
student should understand ... f . Participants who persist in the latter mode need 
to be guided into thinking in terms of the former as the first requirement is 
to ascertain what is the work that an operator must be able to perform. 

Once the technique becomes familiar to participants, cards with skill 
definitions will be produced more rapidly. These cards need to be placed on 
the wall from the right in line with the related duty as quickly as possible. 

Sometimes the facilitator will have to help with the wording. During this 
stage the workshop becomes very much a brainstorming activity which 
continues for most of the remainder of the workshop. During this 
brainstorming session, participants may want to start sequencing skills (cards) 
and contributing skill definitions for other groups of duties. Both these 
divergences should be avoided. 

It is important that the facilitator ensure all participants are contributing at 
this stage in as equal a way as appropriate because the pattern established at 
this early stage of the process will generally persevere throughout the 
workshop. As well, the facilitator needs to keep participants focused on the 
point and should encourage any discussions to continue as long as possible so 
long as they appear to be productive. The group should not break until the 
analysis for this group of duties is complete. 

Analysing the remaining groups of duties 

The facilitator continues the above process for each of the remaining 
duties. The sequence of tackling the duties is not crucial, but it could be wise 
to tackle duties requiring similar or related kinds of skills in groups. That is 
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to say, manipulative or hand skill duties could be completed before confronting 
those duties requiring analytical skills. Individual duties should be completed 
independently. 

Definitions already on the wall can be refined or modified as the process 
continues, but the facilitator should insist on agreement about such changes 
amongst the group. However, the main concern for the facilitator throughout 
this pait of the process should be keeping it moving as quickly as possible, and 
by providing guidance when the group bogs down. 

Reviewing and refining the analysis 

Once it is determined that coverage of all duties is complete, the facilitator 
should lead participants through a quick review of each of the skills and duties 
on the chart. The main purpose of this exercise is to improve the quality of 
definition of skills. No attempt should be made to restructure the chart at 
this time. 

It is important that every card be critically reviewed individually because 
often participants may have refrained from disagreement at an earlier stage 
for the sake of process continuity, or skills may have been poorly defined due 
to lack of expertise earlier on in the process. This review panel should strive 
for group consensus on all the definitions in the chart. 

Sequencing the skills 

A suitable set of criteria for sequencing the skills must be developed by the 
facilitator/organiser and explained to the participants. In keeping with the 
rationale of DACUM, participants can be asked to imagine that they are the 
sole experts in a training situation and are responsible for organising the 
training schedule of a naive (untrained) student. Taking one group of duties at 
a time, participants are asked to sequence skills in the order that the student 
should learn them in order to realise an immediate return on their investment. 
Skills that commonly occur in the workplace would be placed first. These 
skills are moved to the left of the chart. As this process continues, those 
skills are slipped to the right where the learner would need to have developed 
a greater awareness of the occupation and where more complex decisions 
relating to skills would be needed. (See Appendix D). 
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Some vertical alignment of skills in groups of duties may also be arranged 
depending upon the course style to be adopted - for example, whether the 
program is modular or subject based. For example, 'safety' skills may appear 
in several groups of duties - if these are aligned vertically they can be readily 
seen to constitute a unit or module or a logical group of learning objectives. 
The nature and complexity of the sequencing required on the DACUM chart is 
variable and depends upon the selection and statement of the sequencing 
criteria by the facilitator/organiser. 

Establishing levels of competence 

This process is often conducted at subsequent meetings of 
teacher/educators, but may if desired be conducted as the last stage of the 
DACUM chart development exercise. 

A simple technique for establishing levels of competence may be used - 
where each skill is identified with one of the three labels 'limited', 'moderate' 
or 'skilled' level of competence. This is considered general enough for DACUM 
workshop participants to agree upon and provides enough information for course 
developers and teachers to operationalise. A simple L, M, or S is recorded on 
each chart. (See Appendix D). 

Final structuring of the chart 

After competence levels have been completed, a final review of the chart is 
undertaken. Minor additions, deletions and modifications are made via 
consensus. 

The facilitator must ensure participants are aware what will happen to the 
chart and how it will be used to design the curriculum. Once the chart is 
produced, copies should be sent to participants and to other appropriate 
people. 
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chairs for observers 



10 metres of unbroken wall space 
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chairs for participants 
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Sign stating: The individual/student must be able to 



Figure 3.! Physical layout of workshop room 



suitable for conduct of a DACUM session 



DACUM modifications: the SKA seminar 

The Skills/Knowledge/Attitudes (SKA) seminar is a group process method 
derived from the DACUM approach. The aim in developing the SKA method 
was to pare the occupational analysis process down to the minimum data 
necessary for curriculum development. SKA combines an occupational analysis 
and a task analysis. It extracts the main categories of work performed in the 
industry and the tasks needed to perform work within those main categories. 
The technique is based on a workshop using industry 'experts', a curriculum 
specialist and a facilitator and teachers. 

The SKA's purpose is to find out what a person does on the job and what 
the student needs to know. The question 'what does the student need to know' 
is included to expand the factual (what is done) data so as to provide 
information about standards of performance which the student should achieve 
and information about the progression or sequencing of learning, though this 
information is collected qualitatively without using the kinds of rating scales 
applied in the DACUM method. 
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Itefere the selection of the workshop participants it mod?, the question 
•***e i» b**in and end the process mutt be decided. For example, should the 
poprt rover the trade portion of tht work only, or iht whole Industry 
spectrum, including knowledge of related areas or processes? Some initial 
etpfc* atian h needed on %h#t the work embraces. 

The workshop process 

The SKA seminar involves up to 24 people chosen as rt»* esentatlves of the 

industry under investigation, working in an intensive workshop of three days 

duration which includes small group and plenary sessions. In the three day 

version the listing of main duties and the tasks necessary to perform those 

duties is usually completed by midday on the second day. Detailing the 
sub-tasks takes the remainder of the time. 

The facilitator need not be a specialist in the subject area, indeed, the 
process is reported to work best when there is not an 'expert' leading the 
workshop. It is most important for the facilitator to show that the input of 
the group is being heard by writing all comments, non-judgement ally, on 
butcher's paper around the wall. The facilitator can spoil the process of 
extracting the main duties by not sticking to the principal question: 'what does 
the person do', and allowing individuals to manipulate or dominate the group by 
convincing the facilitator that they are describing what the worker needs to 
know rather than describing what the worker actually does. 

To build a sense of group cohesion, the group can be brought to the 
workshop venue, e«g. a motel with conference facilities for a meeting on the 
evening before the workshop, where they are given an overview of what the 
workshop will entail. Users of the method advise that it Is most Important to 
begin and end with the same group. People who drop out should not return, 
m the small group work a ratio of one teacher to five or six industry 
representatives Is regarded as Ideal. 

The wor k shop or seminar begins with small groups working on the 
Identification of the large pieces of work' (Le. the main work categories 
performed In the Industry). An example of a main duty for fitting and 
machining Is "operate a lathe'. The focus during the establishment of the 'main 
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duties 1 on 'what a person does 1 Is achieved by the use of the phrase 'what does 
the person do? 1 not 'what does the person need to know'. After the main 
duties have been stuted und listed on butcher's paper in small group sessions, 
the groups come Into plenary session for the establishment of consensus. This 
process usually collapses the number of main duties Into about eight to ten. 
The duties comprise various tasks which can be further divided Into sub-tasks. 
Some sub-tasks have elements (smaller sub-tasks). The laying out of the 
sub-tasks, and there can be up to thirty sub-tasks, Is done In small working 
groups of 5-6 people and usually takes about 2-3 hours. The next step is to 
select one or two main duties and extract the tasks that need to be known in 
order to perform the work within the specified main duty. The emphasis in 
extracting tasks Is first on 'what the person does' then on 'what does the 
person need to know' (to perform the piece of work). This work Is written up 
on butcher's paper. 

Shortened form 

The SKA process can be shortened to a one-day workshop, which ceases at 
the stage of eliciting duties and tasks, and leaves out detailed specification of 
sub-tasks and what the student needs to know. 

Validation 

In both the long and short form, the results can be checked by doing a 
survey but this Is reported to be very time consuming. Another option is to 
conduct a survey on the final syllabus, rather than on the results of the SKA 
workshop. 

For the SKA method to be successful, it is said to be most important to get 
the right people at the right time and not to use part-time teachers. 

Data collation and checking 

After collection of data, some data translation may be needed. This is 
usually done by two people, a teacher and a curriculum developer. A scalar 
diagram is developed which sets out the main duties, tasks and sub-tasks and, 
if needed, the elements which make up the sub-tasks. A scalar diagram is 
essentially an attempt to present data in a logical progression, hierarchy or 
'tree' form, e.g. the main duties are grouped with the tasks and sub-tasks, 



ERIC 



needed to be known to perform them* The scalar diagram is typed, then 
checked by a small group of say, five people, and a small survey may be 
conducted for verification. The process can take three months including 
writing a syllabus document. 

Format and steps for a three day SKA workshop 

Step 1. Hold a short introductory meeting over drinks the night before the 
workshop in which the facilitator begins by explaining 'This is what you're here 
for, this is what we will be doing, this is how we will be working.' The 
facilitator states that the major work areas of the occupation will be identified 
then these will be broken down into their parts. For example: for the work of 
hanging a picture (major area) the task level includes putting a nail in the 
wall, attaching a wire to the picture and hanging the picture on the wall. The 
next step is to ask what is involved in putting the nail in the wall (locate the 
wall stud). Then the question is asked 'what are the skills you want people to 
have, for example, in locating a stud, and what do you need to know to do 
that'. 

Step 2 (Workshop). Work begins first on the major areas of (main) duties. 
Working groups of 4-5 people are used then a plenary session is convened to 
get agreement that they are indeed the duties. The process depends on how 
many duties are listed. Usually three to four quite distinct areas are named. 
Again, small groups are used to look at each particular duty/task area and to 
list the sub-tasks and, if needed, the task elements. 

The timing of the process typically breaks down as follows: naming duties 
(1.5 hours); naming tasks (1.5 hours) and then a break for lunch; listing the 
sub-tasks and the task elements (7 hours). 

Step 3. After the workshop, the teachers do a cleaning up of the data to 
get them into the right format and also to look at other background questions 
which might be asked. 

Discussion 

A problem with the SKA process which applies to the DACUM method in 
general is that in revision of large courses were the main body of teachers 
have been excluded from the workshop, the results may be resisted because the 



102 1U 



results may not support retention of certain aspects of the current course. 
Yet the process is said to bring traditional subject content, e.g. mathematics, 
into line with what is needed by the student to perform the work. It appears 
that the SKA process can show up possible areas of overtraining, though tension 
can arise between what industry says is needed as opposed to the extra 
knowledge educationalists believe the student ought to have for reasons of 
increasing, say, problem-solving ability, mobility within the occupational field 
and so on. For this reason it may be useful to use DACUM as part of a blend 
of methods (see Chapter 4). Accounts of the SKA process indicate that the 
people who have participated, like it. If the course objectives are correctly 
written, these form the basis of assessment and teaching/learning resource 
development. 

Two adaptations of DACUM 

The following versions of DACUM have been used in Australian TAFE. 
Version 1. 



1. Discuss the project with representatives from 
the teaching body to break down major duties 
(at a basic level), 

2. give an introduction to DACUM and allocate one duty 
area to each teaching college or section, 

3. bring together a representation of employers and colleges 

in a DACUM session. The representatives in a DACUM session 
include a cross-section of interested parties, e.g. 
employers, supervisors (working in the industry), 
practitioners, teachers, union representatives, 
curriculum developers (including two people trained in 
the DACUM method). 

4. Teachers produce a flowchart giving a profile of 
competencies needed for a student entering the 
industry, including the knowledge and skills needed. 
Connections between the lists of duties should be teased 
out (e.g. the need to be able to set priorities) and 

the most basic skills should be listed that go 

toward realisation of competency in the main duty areas. 

The 'main duty' flowchart lists main duties vertically 

and displays horizontally, the knowledge/skills 

needed to acquire competency in each main duty area. 

The arrangement of knowledge/skills groups (reading 

from left to right on the flowchart) can be viewed 

vertically to note important sequencing for learning. 
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5. As in the DACUM method a ranking procedure can 
be used to Indicate the depth of learning needed, 
A 0-6 numbering system can be used for this, and a 
second numbering system can be used which gives 
information of levels for assessment. 



An adaptation of DACUM which has been used in Western Australia proceeds 
as follows. Teachers are excluded from the workshop. Their role begins when 
they are handed the data from the workshop. For example, for the 
mathematical task of 'applying third angle projection 9 the task statement will 
be provided to teachers in the form: 'Task: students would be able to apply 
third angle projection'. The teachers are asked to write this in a behavioural 
objective format, e.g. 'The student must be able to do the following...' 

The teachers work through each of the tasks produced by the workshop and 
for each task are asked: 'Do you agree that this is what the person does?' 
The time allocation required to effectively teach the task is written beside the 
task. When all the tasks have been examined and a probable duration for 
teaching has been estimated the full set of tasks is examined in sequence and 
a total estimated time to teach them is calculated. The next step is to 
describe each substantive task area in synopsis form and to hand these to the 
subject area specialists for writing the detailed objectives. It has been stated 
that the detailed syllabus can be written within a matter of days. Note that 
the depth of treatment for each task is decided by the teachers. The final 
step in the process is to send the completed questionnaire back to industry for 
confirmation. 

3.3 THE CRITICAL INCIDENT TECHNIOUE 
Background 

The critical incident technique was developed by Flannagan in 1947 in 
response to a problem of pilots losing their lives by crashing military aircraft. 
The technique is described in Flannagan (1954). Flannagan got together a group 
of pilots, those who had flown the aircraft and still lived, and asked, in 
essence, what do you have to do to fly these aircraft without crashing them, 
and what must you not do or you will crash them? That is, he was looking 
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for the 'do's' and 'dont's' - the sort of trial and error learning necessary for 
flight survival that was not in the flight operator manuals. 

Ulshack (1983:133) states that the effective assessment of training needs 
'should formulate a structure upon which training programs can effect a 
behavioural impact 1 . This is taken to mean that the underlying concepts on 
which the learning program is intended to have an impact must be 
formulated. The critical incident technique, he notes, 'has perhaps the most 
direct behavioural link of any method of needs assessment'. The method 
consists of a set of procedures for collecting direct descriptions of human 
behaviour in a relatively systematic and objective manner (Noesjirwan et al. 
1979). 

Such descriptions can include self-reports of recalled incidents by use of a 
free-ranging interview in which the person is encouraged to talk about 'critical 
incidents' that happened to them and the meaning that these had for them, 
including the effect upon them, and what they did in response (Noesjirwan et 
al. 1979). Descriptions can also include identification of critical problem areas 
and systematic identification of skills required to deal with these problems. 
Data from observers can be used to supplement self-reports or to identify 
critical problem areas and skills needed. 

The outcome 'is a description of observable behaviours that are "critical" to 
the effective performance in a defined activity' (Ulshack, 1983:134). The 
method assumes that jobs are composed of critical and non-critical tasks. The 
emphasis on 'critical incidents' alone can be a problem. Critical incidents may 
emphasise the problematic at the expense of the important but routine tasks. 
A critical task is a completed action sequence which makes the difference 
between success and failure in carrying out important parts of the job. 
Non-critical behaviours relate to tasks which have little relation to success in 
securing the overall completion of the job or are done so well by workers that 
they are not made a source of comparison between one worker and another 
(Youngman et al. 1978). 

The relevance of the behaviours involved in critical incidents means that the 
critical incident technique can be used as the basis for inferring the human 
qualities that are relevant to successful performance (McCormick, 1979). The 
method provides an 'ideal basis for the development of training programs and 
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training materials 1 (Flannagan, 1954:307), Moreover, a major contribution that 
the critical incident technique can make to curriculum content identification is 
its 'potential to deal more directly with isolating important values and 
attitudes" (Finch and Crunkilton, 1979:129). Using the technique 'one can select 
those behaviours which are attitude o value-laden and thus provide a firmer 
foundation for affective content in the curriculum'. 'Typically 100 to 200 
incidents are gathered then grouped into categories with general headings,' 
These 'serve as a foundation for curriculum content that focuses on developing 
appropriate attitudes and values' (Finch and Crunkilton, 1979:129-130). Ulshack 
notes that where a large sample of critical incidents have been collected, a 
functional description of a job can be derived. The method has various 
applications such as 'criteria development, measures of proficiency selection, 
job design and purification, establishing operating procedures, equipment design, 
motivation and leadership, as well as training' (Ulshack, 1983:134). 

Support for the use of the method for performance appraisal was provided 
by Levine et al. (1981:43). They conducted a comparative study of seven job 
analysis methods and found that the critical incident technique was 'best for 
personnel selection, compared with: Threshold Traits Analysis, Ability 
Requirement Scales, Position Analysis Questionnaire (PAQ), Task Inventory 
compared with the CODAP computer software package, Functional Job Analysis 
(FJA) and Job Elements (see Appendix A). They note that 'the critical incident 
technique is central in the development of job performance measures 
(behaviourally anchored rating scales) but that the method was the most 
expensive to apply'. 

Zemke (1979:67) describes critical incidents as: 

reports or descriptions of behaviours enacted by people in the 
population being studied. These incidents are recorded and 
classified as effective or ineffective in achieving the desired job 
results. The descriptions can take the form of stories, anecdotes, 
reports and observations related verbally or in writing, by 
supervisors, peers, subordinates or any other observer qualified to 
judge the performance. 

As noted previously, the method draws upon practical 'trial-and-error' 
knowledge of crucial performance do's and dont's. The gathering of 'critical 
incidents' can be done by observers such as researchers or supervisors or by the 
job incumbents themselves. According to McCormick (1979:138) the most 
complete form of recording includes: 1. What led up to the incident. 



ERLC 



106 

11K 



2. Exactly what the incumbent did that was so effective (or ineffective). 3, 
Perceived consequences of the behaviour. 4. Whether consequences were within 
the control of the job incumbent. 

Ulshack (1983:135-6) gives four principles vital to the success of the critical 
incident approach: 

A. Only simple judgments (success or failure) are required of 
the observers. B. Only qualified observers - supervisors, peers or 
incumbents of the job - should be used. C. Incidents must 
represent behaviours that have consequences directly attributable 
to them. The behaviour can be identified as causal to the 
outcome. D. Specified requirements or standards for the incidents 
are established which might include requirements ?jch as: (1) the 
incident should be observed first-hand; (2) the incident reflects 
aspects of the behaviour, not the individual or his personality; (3) 
the incident has occurred within a specified recency (e.g. the last 
six months). 

Note that the technique's basis is observable behaviour. This is believed to 
maximise objectivity while reducing subjective components of inference and 
interpretation (Ulshack, 1983). 

Main steps in applying the method 

Aims 



The aim of a critical incident project is to find the completed action 
sequences (incidents) that are critical for the performance of the job, either 
positively or negatively (i.e., the do's and dont's). 

Critical incidents must address four questions. 



1. What was done that led to effective job performance? 

2. What was done that detracted from effective job 
performance or led to ineffective job performance? 

3. What, if done prematurely, would have been more 
effective if done later (indicating critical 
problems in planning)? 

4. What attitudes, values, abilities, knowledge, skills 
(present or absent) seemed to lead to success or failure? 

2, Sample and questionnaire 

The sample can be any group having the opportunity to directly observe the 
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incumbents performance (e.g., the incumbent, peers, supervisors). A method of 
data collection proposed by Ulshack (1983) is as follows: 

1. Recall. Respondents describe critical incidents in 
writing or verbally. 

2. Observation. An observer records critical 
incidents (do's and dont's). A 'co-researcher' 
approach would be useful here, where the observer and 
a highly skilled incumbent, together research the 

do's and dont's of performing the task(s). Film 
or videotape may assist the analysis and provide useful 
teaching materials. (See details on instructions for 
observers, below.) 

3. Interviews. The procedure of probing for critical 
behavioural incidents by interview is an effective 
method for drawing out knowledge that an incumbent 
has built up through trial-and-error learning about 
how to perform a task effectively. Interviews can look 
for what is salient (critical) in performing a task 

and for the decision rules, in terms of do's and 

dont's, which must be applied to perform a task effectively. 

4. Questionnaire. Where the respondent group is large 
and dispersed, a questionnaire approach may be used 

as in the following sample quoted from Ulshack (1983:139): 



'A. Think of the most difficult situation that you have handled 
successfully because of your training as a veterinarian. 
Briefly describe the situation. 

What training (specific class or seminary) helped you to 
appropriately respond? 

How recent to the incident did the training occur? 
What was the outcome of your action? 

Why do you feel this was the most difficult situation you have faced? 

B. Think of the most difficult situation you have handled which 
was unsuccessful because of a lack of (or poor) training. Briefly 
describe the situation. 

Have you faced the situation since? 

Do you now have the appropriate way of handling the situation? 

How did you learn the appropriate treatment? 

What was the specific outcome of the unsuccessful handling of 
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the situation* 



How would you recommend training other veterinarians to handle 
the situation 1 * 



3. Instructions to observers 

Instructions given to observers should include seven specific items (Ulshack, 
1983:140-1): 

1. Explain the general purpose of the project. 2. Assure 
anonymity of the observers and incumbents. 3. Explain that an 
incident is an observed specific behaviour which led to a specific 
outcome which was either a success or a failure. 4. An effective 
optional step is to give observers an example of a critical 
incident, but make sure that the example is either positive or 
negative. 5. Define precisely what you want the observers to 
report. 6. Specify time limits, job or situation to be observed, 
emphasise the necessity of actual observations of behaviour, and 
that the behaviour should be a completed action with a result. 7. 
Lay out the mechanics of the collection data phase, i.e. when, 
where, and how incidents will be collected. 

4. Data analysis 

The main steps in data analysis are summed up by Ulshack (1983:142) as: 

1. Decide the frame of reference: how is the data to be used 
in training; what are the training goals? 2. In the formation of 
categories the main considerations are: what are the critical 
behavioural dimensions, how do these group together, how can they 
be turned into descriptive statements of behaviours to be 
performed? 

Ulshack advises that the critical data analysis be tested on independent 
analysts to check if any categories of behaviour are inappropriately defined. 

A procedure for critical incident analysis, developed by Zemke (1976) is: 

1. Form two groups of expert analysts. 2. Describe in writing 
the specific job incidents and the results produced. 3. Have the 
experts sort the incidents into factors, neme the factors, then sort 
the incidents for 'goodness 1 and 'badness 1 along a 7-point scale. 4. 
Have the experts review the data and fill in any gaps. 5. Have 
the two groups swap results and reconcile any differences. 

After a group of incumbents have passed through the critical incident 
procedure, it may be useful to hold a group discussion aimed at bringing out: 
general information about the work context and its relationship to task 
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performance, and specific information about knowledge and skills needed fc 
the development of an educational program. Here, the techniques offered b 
DACUM or NGT may be usefully applied. 

5. Sum i nary of advantages and disadvantages 

Some of the advantages of the method are summed up by Ulshack (1983): 

1. It develops behaviourally-based criteria. 2. The method is 
objective. 3, It takes account of the multi-dimensionality of 
performance and enables incidents to be categorised into a 
structure that should represent the natural job dimensions. 4. It 
provides reliable, relevant criteria. 5. It provides comprehensive 
information. 6. Incidents may be translated into behavioural ly 
based performance appraisal instruments. 

Some disadvantages include: 

1. The role of observers. When observers are used the 
method requires considerable sensitivity and skill in 
observational analysis. The observer's role as 'watcher 1 
may disturb respondents, especially if they feel they 
are being judged. The co-researcher approach mentioned 
previously could be useful. In this approach the person 
being studied is invited to become a partner in the 
research (hence 'co-researcher') with the curriculum 
developer providing key concepts, then working with 
the person to fill these in or to expand them. 

2. The formation of categories can be time consuming in 
planning and implementation and does not cover routine 
job dimensions. Here, DACUM would appear to do a better 
job. 

A problem which can arise with the method is how to define 'critical'. In 2 
application of the procedure for study of teaching methods, Killen et a 
(1983) chose to define 'critical' as posing a need for intervention. Note that i 
any application of the procedure there must be careful preparation of tti 
initial list of incidents (Grant, 1979:498). 

Conclusions 

The technique has promise because it brings out information about gaps i 
knowledge held and knowledge needed on-the-job. It is well suited to a diar 
approach to data collection and would appear to have application graduat 
student surveys which aim to assess the utility of an educational program. 
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The technique is flexible, relatively simple, and easy for respondents to 
understand. The centrality of critical incidents to the performance of tasks 
facilitates recall and description, since these are the events that stand out in 
memory. The data which the method provides are the crucial dimensions for 
effective performance of tasks. In the hands of a skilled analyst who will take 
the time to probe behind these dimensions, this data can have utility in 
examining the factors which shape the problems which lead to critical 
incidents. So, on one hand the method can lead to redesign of equipment, and 
on the other hand, can lead to a better understanding of how to work within 
the limitations of equipment. The method would appear to be useful in 
'trouble-shooting' on specific problems, both technical, e.g. what are the 
critical incidents in performing a particular process, and social, e.g. what are 
the critical incidents in successful social performance such as customer 
relations work or staff supervision. In technical applications, due to the 
experiential knowledge drawn upon in identifying critical incidents and the 
fine-grain data which emerges, the method would lend itself to a) the 
development of models describing decision-making processes and b) the 
development of teaching strategies and materials aimed at transmitting key 
concepts about the effective performance of tasks. 

The method appears to have wide applications in TAFE. Ape r * from its use 
in occupational analysis, teachers can use the method to review instructional 
methods by gathering 'critical incidents' data from students. The method also 
permits student self assessment (on the basis of success in performance of 
critical incidents) (Grant, 1979:498). 

As to cost and duration, little information is yet available on TAFE 
applications of the method. However, in relation to cost, Levine, et al (1981), 
as noted earlier, describe the method as costly to apply, although they did not 
comment on its cost effectiveness (cost of application versus quality of data 
produced). 

3,4 FORCE FIELD ANALYSIS 

Force Field Analysis has its roots in Lewin's (1952) development of Field 
Theory which was formulated on the basic principle that 'any behaviour or any 
other change in a psychological field depends only upon the psychological field 
at that time' (1952:45). Lewin used the concept of 'valency' to refer to the 
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pull or attractiveness of an object in the perceived field. He felt that the 
method for solving problems should be one that sees the problem as a balance 
between opposing forces of attraction or repulsion. Some forces help the 
person's movement toward a desired state and others restrain such movement. 
Using these ideas, a method called Force Field Analysis has been evolved. 

Force Field Analysis is a technique for 'analysing change situations' (Eiben, 
1976). A force field is the total of all forces, positive and negative, that 
affect a particular situation. The basic notion in force field analysis has been 
described by Johnson and Johnson (1982) as follows: 



Helping forces . Restraining forces 



(-) 

worst 
state 



(+) 

ideal 
state 



current state 
of affairs 

Figure 3.2 Force Field Analysis 

The ideal state of affairs toward which the group is working is shown on the 
right-hand side of the figure by the plus sign and the worst state of affairs on 
the left side of the figure which is shown by the minus sign. The basic steps 
in doing a Force Field Analysis are: (1) In a group-setting use 'brainstorming' to 
develop, on a large sheet of paper, a list of the helping and the restraining 
forces. These should include the full tapestry of possible forces: psychological, 
sociological, etc. In the normal rules of brainstorming, all criticism is withheld 
and every member's ideas should be requested and noted without judgement. 
(2) The forces are then ranked according to the importance of their effect on 
the present situation. The strength of the forces can be depicted by the 
length of the arrows (Spier, 1972:113). (3) Once the important helping and 
restraining forces are identified, these are rated according to how easily these 
can be increased or decreased. The method then proceeds by taking those 
items which are easiest to change with a likelihood of success. The more 
difficult problems can then be tackled. This method helps to clarify the set of 
strategies which must be mounted to achieve the goal. Some strategies will 
focus on increasing the 'helping' forces and other will focus on weakening the 
'restraining' forces. Some may work on both 'helping' and 'restraining' forces. 
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Main steps in applying Force Field Analysis 



Eiben (1976) gives the steps in performing a Force Field Analysis 
as: 

1. Identify the problem - in terms of the forces that 
provide both positive and negative impetus to its solution. 

2. Ask individuals to identify as many problems or areas 

of concern for the problem. Identify the forces which might 
assist in the solution of the problem and the forces which 
might hinder the problem-solution process. 

This can be achieved using the following guide: 



Problem or areas Forces related to this problem 

of concern or concern 



1. 


Helping forces 


Hindering forces 


1. 


1. 




2. 


2. 




3. 


3. 




4. 


4. 




5. 


5. 


2. 


1. 


1. 




2. 


2. 




3. 


3. 




4. 


4. __ 




5. 


5. 



The next step is to form two groups of people of 4-8 members and from the 
individual worksheets (above) list all the problems or areas of concern. Next, 
identify the four or five most frequently mentioned problems and for each 
problem area, identify the helping and hindering forces. The final step is to 
rank these helping and hindering forces from most to least important. 

In order to develop a program of planned change, Eiben (1976) suggests the 
following steps: 



1. Write a goal statement, e.g. to achieve X by Y time. 

2. Use Force Field Analysis to establish the forces which 
promote change and resist change. 

3. Identify as many resources you can relate to the 
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•enlevement of the goal (include Human' and 
•Meier i »r resources). 

4. Develop strategies, ue. interventions that are used 

by the agent of change to causa movement toward the 
goal • by brainstorming. 

5. Implement the strategies. 

Another e*ampl« of how the method can be applied has been provided by 
Utt (1963) who used the following means for establishing the interest within an 
organisation for the adoption of a particular innovation. 



Interest scale 



interest 
Driving forces for 
increased interest 



Organisational 



High Interest 
Restraining forces against 
increased Interest 



Distasteful 
subject 



The driving forces (organisational request) are shown at left and the restraining 
forces (distasteful subject) are shown at right. Once the resistances to change 
are listed, either the push forces can be increased or the restraining forces can 
be reduced, or both sides of the problem can be worked on simultaneously. 
Force Field Analysis 'can give some Insights into the positive and negative 
forces which are constantly at work to either help or hinder training programs 
(Utt, 1981:63). 



The concepts used In Force Field Analysis are similar to those used in the 
Search Conference Technique which also addresses ideal states (desirable 
futures) and restraining forces (constraints). Force Field Analysis would be a 
useful technique to apply In any workshop situation where the group becomes 
"bogged down* In tension or conflict. Along with the Nominal Croup Technique 
the method would be useful In sorting out the central issues where conflict is 
wpsctod to exist, e.g. between teachers about a course program. This 
method would bring out the dimensions of the problem and perhaps the solution 
as well. 
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Although not a curriculum development process In Its own right, the method 
appears to be a useful conflict resolution procedure which can be used within a 
Search Conference, Nominal Group Technique or DACUM process, or in teacher 
workshops where conflict resolution and strategy generation Is required. 

3.5 BRAINSTORMING 

A number of the group process methods discussed rely on 'brainstorming 1 . 
The technique of generating successful 'brainstorming' sessions Is helped by 
observing the following steps (Simon et al. 1976:54-55). 

I. There should be no evaluation of any kind while Ideas are 
being thought up. 2. Everyone is encouraged to think up as many 
wild Ideas as possible. (It is easier to tame down a wild idea, 
than to pep up a dull one). If no wild ideas are forthcoming, 
Individuals are probably 'censuring' ideas. 3. Quantity Is 
encouraged as quantity may breed quality. 4. Everyone is 
encouraged to build upon or modify the ideas of others. 
Combinations or modifications of Ideas lead to new, superior ideas. 

A key to success In 'brain stroming' is persistence. For example, a 
telephone overhead transmission maintenance problem (snow weighing down the 
lines threatening to break them) was solved by the three hundred and fifteenth 
suggestion which was to fly a helicopter over the liines to blow away the 
snow. 
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CHAPTER 4 

WAYS OF BLENDING CURRICULUM RESEARCH METHODS 



4.1 ISSUES RELATING TO ANALYSIS OF COURSE CONTENT 

In Section 1.3 we raised a number of issues relating to thj selection of 
research methods. These issues included: (A) Identifying the 'what 1 versus the 
'how 1 of curriculum; (B) The emphasis to be placed on learning skills versus 
learning concepts underlying the skills, and (C) The problems of analysing 
future directions versus the 'here and now of work in an industry. Here we 
group the methods reviewed earlier according to their potential for addressing 
the issues listed above. In Section 4.2 we present some other promising ways 
in which the various methods might be used in combination. The issues raised 
in Section 1.3 concerning research methodology are discussed in Chapter 6. 

A. Methods having potential for analysing the 'what' versus the 'how' of 
curriculum. 

To find course content (the 'what' of curriculum) consider: 

Preliminary research 
Exploratory research 
DACUM 

Search Conference 

Nominal Group Technique 

Critical Incident Technique 

(Or any other recognised research methods) 

To find how the content could be delivered consider: 



Literature search to identify problem areas and solutions. 
Review of research data by teachers. If teacher workshops 
are used consider using the Nominal Group Technique 
as part of a focused group discussion and/or Force 
Field Analysis to generate strategies for implementing 
solutions. 

B. Methods with potential for analysing the emphasis to be placed on skills tp 
be learned versus learning underlying concepts. 

To identify skills to be learned consider: 
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Preliminary research 
Exploratory research 
DACUM 

Search Conference 
Nominal Group Technique 
Critical Incident Technique 
(Or other methods) 

Identifying underlying concepts. 



Literature search. Review of research data by teachers 

and or/teacher workshops, possibly using Nominal 

Group Technique where the NGT question asks "What underlying 

concepts are necessary to explain how to perform skill X," 

Exploratory research applied to the study of skill masters. 

Use of the Critical Incident Technique, 

C. Methods with potential for assessing future directions versus the 'here and 
now* of work in an occupation. 

The three major considerations in deciding whether blends of methods are 
indicated are: (1) Is the occupational area relatively stable? (2) Is the 
occupational area undergoing change? (3) What pathway should the researchers 
follow in order to understand the occupational field sufficiently to know when 
the point has been reached for course design to begin? 

If the occupational area is relatively stable consider: 



DACUM 

Critical Incident Technique 

Nominal Group Technique (using a cross-section of teachers 
and industry representatives) 

If the occupation is undergoing substantial change it will be necessary to 
identify the nature and extent of the change including skills in learning how to 
learn, ascertain what knowledge and skills are needed to cope with the change, 
and decide what format should the course have in general terms? Consider: 



Preliminary research. 

Exploratory research possibly leading into a 
Search Conference or Delphi to detect the nature and 
extent of change, then decide between a modified DACUM 
in which the participants focus in particular on skills needed 
to cope with change, or use a Critical Incident Technique 
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to identify emerging skills which skilled operators believe 
it will be necessary to master in view of forthcoming change. 
Decisions on course format could then be decided in 
teacher workshops. 

4.2 MULTI-METHOD APPROACHES TO CURRICULUM RESEARCH 

One of the more difficult decisions for a curriculum research and 
development team faced with a 'fast-response' brief is to decide what is the 
necessary and adequate information on which to base sound curriculum 
developer. This question depends, of course, on the specific situation. We 
begin the discussion of blending methods by considering a multi-method 
program for mounting a 'fast-response' curriculum research project. 

Assuming that resources were available and that a curriculum research team 
consisting of one curriculum research and development specialist and two 
subject specialist teachers were faced with an urgent need to develop a quality 
curriculum product which could be well supported with a data-based argument 
(e.g. for accreditation or resource allocation purposes), how might the project 
proceed? 

Remembering that we are looking at an 'ideal' program with a quality 
product in mind, done in a short time frame, the steps in the research program 
could cover: 



. Listing of issues and development of research 
questions (2 days). 

. Preliminary investigation of available information 
including initiation of literature search. (2 weeks: 
1 full-time researcher). Note: Allow for possible delays 
in obtaining literature sources. 

. Decide whether appropriate to survey teachers and 
students. (If teachers, ask in particular what they 
want to see changed. If students, then obtain information 
in particular, on what was relevant and not relevant to 
their on-the-job performance, obtaining relevant background 
information to sort this information by employee category, 
job performed, and so on.) 

. Refine research questions for exploratory research phase 
(2 days: team). 

. Conduct exploratory research (2 weeks: team, full-time). 
During this and the previous stages, key informants 
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with knowledge of the industry would be identified for 
possible inclusion in a curriculum research group process 
method. 

. Review data and: Write summaries in point form for each 
research question from the exploratory interviews, 
review main patterns/disjunctures, decide the appropriate 
curriculum research group process to use, and select 
participants and venue (2 weeks: team). 

. Implement curriculum research group process method. Note 
that two group process methods may be indicated: one 
examining the perceived impact of the future on the 
occupational area (e.g. Search Conference) and one examining 
the 'here and now' of work in the occupation (DACUM-type 
process), or possibly a blend of these. 

. Review results to decide whether the data obtained 
warrant proceeding to syllabus construction and 
(depending on TAFE procedures) whether validation 
is required (2 weeks: team). 

4.3 OTHER COMBINATIONS OF METHODS 

This section describes a range of other possible curriculum research paths 
which are slanted toward the 'fast response' end of the curriculum research 
continuum. The methods range from the broad-brush to the specific. The 
thrust of the broad-brush approaches is to examine trends, dynamics, shifts and 
their implications for: 

1. Course aims and content, including the relationship 
between theory and practical instruction and the 
weights of emphasis for, and within, subjects or units. 

2. The course pattern and structure (i.e. subject grouping 
and duration and the sequencing of subjects throughout 
the course. 

In a curriculum development exercise where 'fast-response' is accorded high 
priority, a broad-brush approach focusing on items (1) and (2) above will 
pinpoint any additional detailed information which is needed. Once the course 
concept is clearly in view a teacher-task force approach could be used to 
obtain the missing detail, if any. 

All the methods discussed here could include a validation step such as a 
questionnaire administered before curriculum development takes place, or a 
pre-test of the course concept by use of a questionnaire or by pilotting the 
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course. These methods are listed below and then developed in more detail in 
the following sections* 



Approach 1. Exploratory research (visits to 'typical applications 1 of the work) 
leading to a questionnaire* 

Approach 2. Exploratory research leading into a DACUM-type workshop. 

Approach 3. Exploratory research leading into a Search Conference plus a 
DACUM workshop held with separate participants. 

Approach 4. Delphi leading into a DACUM or vice versa. 

Version 1» Use of a Delphi (questionnaire, feed-back) method examining 
trends in relation to work processes and skills likely to be needed, leading, if 



necessary, to a DACUM-type workshop examining the duties and tasks 



development. 

Version 2. Use of a DACUM-type workshop to describe the processes, tasks, 
skills (in the 'here and now') leading into a Delphi process to examine the 
likely trends in relation to current processes used. Here the Delphi 
questionnaire technique, or a standard (no feedback) questionnaire would be 
used as a form of validation which also seeks to expand the data to enable a 
more future oriented curriculum to be developed. 

Approach 5. A questionnaire leading to a Search Conference. 

Approach 6. A Search Conference leading into a DACUM-type workshop with 
different participants. 

Approach 7. A Search Conference/DACUM-type workshop blend. 

Approach 8. A combination approach aiming at a more thorough occupational 
analysis. 



Approaches 1 to 8 will now be discussed in more detail. In considering 
blends of methods the reader should keep in mind the utility of the Critical 



performed in industry in the 



'here and now 1 , followed by curriculurti 



Approach 9. Observational studies of highly skilled operators in key work 
areas. (See Appendix C). ! 
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Incident Technique for assessing conceptual and practical knowledge needed for 
successful job performance and the utility of Force Field Analysis for resolving 
conflict and for generating action strategies for implementing or change. 
These methods can be woven into any multi-method mix. Note that many 
combinations of research methods are useful to suit specific situations. 

Approach 1. Exploratory research leading to a questionnaire. 

In this method, exploratory research is used to develop a course proposal 
which is tested on the industry. For an example, see Anderson (1985) for 
which study the questionnaire was in three parts: Part one addressed work 
processes and applications, part two addressed knowledge and skills needed on 
the job, and part three comprised a 'pre-test' of tentative course proposals. 

Step 1: Clarify issues and draft 'research questions 1 (broad questions which 
will be probed during interviews (e.g. 'How is new technology altering the 
pattern of skills needed to perform the work?' 'How is the industry structured 
(how is the work divided within the industry)?' 

Step 2: Conduct interviews exploring the set of research questions and other 
matters that arise and observe work performed. Sampling cases are as many 
'typical applications' as are needed to obtain a cross-sectional view of the 
occupation. The limit of cases may be as little as 4-8 in a small, specialised 
work area). Add such extra cases to the sample as needed to obtain a picture 
of the diversity of the industry. Step 3: Simultaneously with the exploratory 
work, the main features of a possible educational program would be worked 
out. These ideas are developed for course structure and content described in 
terms of educational outcomes. Step 4: Prepare a questionnaire examining: 

. the types of work in the industry and processes and 
equipment used; 

. the knowledge and skills needed to perform the work; 

. the (draft) course proposal, described in terms of aims, 
outcomes, patterns and duration, and, indication of the 
demand for such a course. 

Step 5: Refine the initial course proposal, develop curriculum and if needed, 
pre-test. 



Approach 2. Exploratory research leading into a DACUM-type workshop. 



The purpose of this method, in addition to generating useful data, is to 
prepare the facilitator of the DACUM-type workshop by providing him/her with 
necessary information about: 

. the diversity of the industry; 

• the future trends and how they are affecting the 
pattern of skills and knowledge which will be 
needed by the student to perform the work; 

. the main competencies needed; 

. the essential or main tasks for which knowledge and j 
skill training should be provided; 

. other information relevant to the curriculum j 
development project which was not known at the outset. ! 

The facilitator can now vary the DACUM-type workshop rule that the 
facilitator does not make an input by switching at appropriate times to the 
role of a workshop member and (gently) probing any strongly held views which 
do not accord with the facilitator's knowledge derived from the exploratory 
work: in essence, it would mean taking a 'devil's advocate role'. A problem 
with this method, if a teacher performed the facilitator role, is that vested 
interests may become woven into the workshop output or be thought to have 
done so, posing problems for acceptability of the data to the teaching group. 
The sampling strategy could be to cover 'typical' applications of the work, 
interviewing staff at various hierarchical levels. 

Approach 3. Exploratory research leading into a Search Conference plus a 
DACUM workshop held with separate participants. 

The data from the exploratory phase would be summarised in terms of 
common themes. The issues identified would be used to design the Searcji 
Conference agenda. The purpose of the Search Conference would be to assess 
future directions that the industry is taking and the implication of these fey 
education. The importance of this step is to ensure that the DACUM process 
will not miss emerging skill needs. The DACUM would also supply the detail 
on knowledge, skills and attitudes. Using separate participants for all phases, 
exploratory work, Search Conference and DACUM, would give a wider 
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cross-section of views. The opportunity to compare 'search' and DACUM output 
for the same industry would aid evaluation of these methods. 

Approach 4. Delphi leading into a DACUM or vice versa. 

Version 1. Delphi leading into a DACUM. 

Step 1. A monitor group comprising a minimum of one curriculum developer 
and one subject specialist conduct a Delphi process with a group of industry 
experts. The Delphi process would explore and obtain agreement from industry 
'experts' about what changes are likely in the future and how these changes 
will be likely to affect the pattern of work and the training needs of the 
industry or parts of the industry and also to identify those parts of the 
industry which are likely to remain relatively unchanged. An 'expert' would be 
defined as anyone who is in a position to offer an informed view about the 
industry. Experts need not be employed in the industry; however, the final 
sample must include industry personnel at various levels, e.g. planners and/or 
administrators, marketing specialists, supervisors and others as indicated by the 
research questions. The time frame considered .in this phase would be the 
short term, e.g. the next five years, although the longer view might also be 
considered. 

Step 2. The questionnaire prepared by the monitor group would ask for 
perceptions related to a few key questions covering specific categories of the 
work of the industry, e.g. machinery, materials, processes, and 
management/supervision. The following questions are intended as general 
examples to illustrate the method. 

1. What are the main changes which are likely to take 
place in the (INSERT WORK CATEGORY) within the 
short-term future, e.g. in the next five years? 

2. What are some of the likely effects of these changes 
on the pattern of work and/or skills required to 
perform the work? 

In answering, please focus on training needs. 

Alternatively, a more specific form of question can be asked which lists 
existing materials and/or processes and asks for a ranking of the suitability of 
the item as either 'outstanding', 'acceptable' or 'not acceptable' to the intended 




use, within a nominated time frame, e.g. the next five years. For this type 
of question to be relevant, some exploratory work would first be needed The 
two methods for doing this are: (1) The teacher(s) conduct plant visits of a 
cross-section of the typical applications of the process and work being studied 
or (2) A DACUM workshop could be conducted to establish the main duties 
performed by operators and the processes used. With this alternative the 
DACUM data would then form the basis of the Delphi questionnaire. That is, 
a 'search 1 would then be conducted on the DACUM data with the aim of 
ascertaining which work processes and attendant knowledge/skill areas are likely 
to undergo significant change in the future. 

Step 3. The monitor group would analyse the Delphi responses and return the 
results to the original expert group for further comment or expansion. At this 
stage the group may be asked to rank the pooled perceptions in order of 
importance or likelihood. This 'feedback 1 process would be repeated a second 
or third time and would end when consensus appeared to have been reached. 

Step 4. At the completion of the Delphi procedure, decisions would be made 
as to what methods would be used next. For exampla, a DACUM workshop (or 
workshops) could be used to develop curricula for those areas of industry which 
appear relatively stable. Another method, such as the Search Conference 
method, could be used to develop the course aims, course structure and topic 
content for courses designed to accommodate future shifts in the industry. 
The central principle underlying such courses would be their flexibility in 
meeting future needs within the industry. 

Version 2. DACUM leading into a Delphi process. 

Here the Delphi questionnaire technique, or standard (no feed-back) 
questionnaire would be used as a form of validation of the DACUM but would 
also seek to expand the data to enable a more future oriented curriculum to 
be developed. 

Approach 5. Questionnaire leading into a Search Conference. 

A short questionnaire would be devised to obtain basic descriptive data about 
work done, by whom, and at what stage of training (or whatever is 
appropriate). This information would then be used in planning and conducting a 
Search Conference. 
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Approach 6. A Search Conference leading intc a DACUM-type workshop with 
different participants. 
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Step 1. Conduct preparatory work (begin listing issues, conduct a literature 
search and so on). 

Step 2. Identify key informants for participation in the conferences: For the 
Search Conference, select participants to achieve a cross-section of informed 
views about the area to be searched, including management and 
practitioner-level industry participants, and other stakeholders. (Total: 25 to 
35). For the DACUM-type workshop, select practitioners and include students 
selected from each stage of the course. Include, say, 10-15 practitioners from 
industry, six students and two or three teachers who will design the 
curriculum. (Total: 18 to 22). 

Step 3. Conduct a Search Conference. The purpose of this conference would 
have four aims: The first would be to explore the forces that will shape the 
future of the industry so that the course aims, content and depth of treatment 
of the content would result in the student obtaining the type of knowledge and 
skills which will be needed upon graduation. The second would be to state the 
thrust of the course aims or to produce the course aims in a more final form. 
The third would be to list the main subject areas which should be included in 
the course, giving some indication of scope, depth and weight of emphasis. 
The last aim would be to describe the essential resources needed for the 
course to be run successfully. 

The conference would begin with a 'future scan', then do a brief historical 
review of the main turning points in the evolution of the industry and in the 
provision of education. It would then cover 'strengths and weaknesses' in the 
current educational system and conclude with development of overall course 
aims and a priority ranking of subject areas to be included with a statement of 
the resources needed for implementation. 

Step 4. Conduct a DACUM-type workshop. The purpose of this workshop 
would be to examine the work currently performed in the industry in terms of: 
main categories of work performed; the tasks/knowledge needed to perform 
work within these main categories (including the sub-task level, as appropriate); 
and the knowledge, skills and attitudes needed to perform the tasks or 
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sub-tasks. 



Step 5 Data analysis. The data analysis phase would entail a synthesis of 
the results of the Search Conference and DACUM workshop by considering the 
gap between the future (Search Conference data) and the 'here and now 1 
(DACUM data), with the aim of producing a future oriented educational 
program and the resources needed for its implementation. 

Approach 7. A Search Conference/DACUM-type workshop blend. 

Step 1. Conduct preparatory work (issues clarification, literature search and 
so on). 

Step 2. Identify workshop participants (aim for representative course-section 
of stakeholders). 

Step & Conduct a two-o a; workshop as follows: 



(a) Future scan (2 hours). Examine the main forces impinging 

on the area. 

(b) Historical reconstruction (1 hour). Examine the main 

turning points in the work, its processes and the 

history of educational provision within TAFE. 

(This is to give credit to the original decisions about 

training and also to allow these decisions to be seen as 

having taken place within a context which has now changed.) 

(c) Examine current strengths and weaknesses (1 hour). 

(d) Examine the industry at the present time. Apply DACUM 

process, extracting the main duties, tasks and 
sub-tasks, needed to be known to perform the main 
duties. This process is intended to lay out the work 
of the industry in sufficiently fine detail to enable 
the next step to take place. 

(e) Relate the DACUM data to items (a), (b), and (c), 

and refine the DACUM data by addition, deletion or 
allocation of weights of emphasis in the light of 
future trends (2 hou s). 

(f) Decide the main thrift op the overall course aims 

bid the main recurem? 'i for structure and content. 
(The extraction o. ?ir \emes may be done in a 
plenary session) (<; . 

These groups would be asked to prepare the main features of a course 'for 
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the future 1 , not just one for the present. The working groups would be asked 
to consider first the ideal course, then to consider those constraints which are 
hindering the ideal, and, in particular, to nominate any resources constraints 
which must be overcome it effective educational provision is to be realised. In 
their final submission to the plenary session the working groups would be asked 
to indicate the features of an ideal course but at the same time to 
concentrate on a course which is a balance between the ideal and the present 
reality. The conference would end at this point to review the data. 

Approach 8. A combination approach aiming at a more thorough occupational 
analysis. 

Assemble a converging program as follows: 

(a) Conduct an exploration by industry visits, interview 

and observation of typical applications 1 of the work. 
Decide on the number of interviews needed according to 
the diversity of the work, and achieve at least one 
visit to each 'typical 1 application, (Note: This is 
an add-on or 'snowball 1 sampling strategy which can 
be started at any point in the sample population with 
cases being added on as the picture of the work 
diversity emerges.) 

(b) Simultaneously with the exploratory work: 

. Identify key informants for the proposed workshops 
i.e. draw up lists of possible participants for Search 
Conference or DACUM workshops. 

. Conduct information search (literature, industry, 
forecasts, syllabus documents, related work in 
progress.) 

. Begin extracting issues to be addressed. For example, 
list data on industry trends, data on strengths and 
weaknesses in the course, industry views of 'need 1 
and 'demand 1 , and form a picture, from the industry 
visits of a course (described in terms of overall aims 
and essential subject areas), which might constitute a 
line-of-best fit, i.e. accommodate industry diversity 
whilst providing the essential theory and practical 
knowledge. (Note: This means that broad-brush course 
design work could begin during the exploratory phase. 
The course (aims, content, scope, depth) 
will become questions to be explored, refined and 
validated thro ^h later phases of the project. 

. Identify further work that should be initiated. For 



example, should the exploratory work land possibly the 
initial broad-brush course Ideas! be turned Into a 
questionnaire at thu s.ege? Decide whether detailed 
observational studies should be done of highly skilled 
operators In the key work areas In order to ascertain 
the top level of knowledge and skills required In 
these key areas? (See Appendix C.) 

Conduct workshops (Search Conference or DACUM-type). 

Decide on validation procedures. For example, trial a 
questionnaire based on exploratory and workshop data 
with or without refined course proposals, on the 
original sample of 'typical* applications plus the 
firms represented by the workshop participants, Ue. 
the workshop participants hand the questionnaire to 
respondents at all relevant levels within their 
organisation. 

Run preliminary data analysis. Decide whether to 
conduct a full-scale questionnaire survey and, if so, 
whether to begin syllabus drafting using the full-scale 
questionnaire results for fine-tuning; or whether to 
delay syllabus drafting until all data are in. 

Develop curriculum. 
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CHAPTER 5 

t 

THE IMPLEMENTATION OF GROUP PROCESS RESEARCH METHODS: 



THE ROLE OF THE FACILITATOR/CROUP LEADER 

5.1 INTRODUCTION 

Gathering qualitative data using curriculum fc^uarch group process methods 
places the facilitator (or group leader) in a crucial position in relation to 
quality of the workshop output. It is therefore important to understand the 
facilitator role in the implementation of curriculum research group process 
methods. This section is a summary of a study of the facilitation of 
curriculum research workshops (Anderson and Jones, 1986) which examined how 
a group of experienced TAFE curriculum developers planned and conducted 
group process methods for TAFE applications. These approaches were analysed 
in terms of the value orientations (preferences for things and actions) anil 
social rules (guides for social behaviour) found to be effective in conducting 
successful curriculum research workshops in TAFE. The intending workshop user 
may wish to refer to the above mentioned study for more detail about the 
complex skills required to conduct group process methods and for an 
explanation of the concepts of values and social rules. The main findings from 
that study are summarised below. 

5.2 GENERAL STAGES IN CURRICULUM RESEARCH WORKSHOPS, AND 
VALUE ORIENTATIONS AND SOCIAL RULES APPROPRIATE TO EACH STAGE 

Table 5.1 gives a general picture of a curriculum research workshop in 
terms of the main stages and steps, and the values and rules appropriate to 
each stage. It is not meant to relate to any particular curriculum research 
group process method but to highlight the general principles which should be 
addressed when designing and implementing curriculum research group process 
methods. The data on which the table were based were derived using a 
modified Delphi process (eliciting value orientations and social rules used in the 
conduct of curriculum workshops), followed by a two-day seminar exploring th^ 
complexities of the facilitator role. This method of leading into a 
workshop/seminar from a Delphi questionnaire had two advantages. First, it 
was an excellent 'warm-up' procedure for genius participants to think ahout 
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the issues. Second, it gave each participant a look at what the other 
respondents saw as the important dimensions of the facilitator role. 



An important point to bear in mind about the concept of social rules is that 
rules can be made, broken and renegotiated (Harre and Secord, 1972). Rules for 
making and breaking the rules of curriculum research and development 
workshops depend on the curriculum research methods being used and on the 
situation in which they are being applied, i.e. on the dynamics of the seminar 
itself. The philosophies of curriculum research methods differ but, in 
implementation, they share as common ground the skills of working with people 
in groups. The purpose of Table 5.1, therefore, is to show the common themes 
for working successfully with people in TAFE curriculum research workshops. 
These themes or value orientations are as follows: 

A humanitarian focus 

Although facilitators vary in the way they approach the task work of the 
seminar or workshop, they appear to be very person centered in their approach 
and attend closely to the personal needs of the workshop participants. It 
appears that when personal needs are fulfilled the task work flows with greater 
smoothness. 

A task-oriented focus 

A good facilitator achieves the task through making the aims, expectations 
and outcomes of the workshop clear, attending to individual and group needs 
sensitively, defusing and/or resolving conflict, watching the use of time and 
using control where appropriate. 

An egalitarian focus and avoidance of 'power-tripping' 

A good facilitator avoids 'power-tripping'. This is important because The 
facilitator possesses a degree of power which, unless handled sensitively, can 
have damaging effects. This power can be expressed in an autocratic, 
overbearing manner used in an attempt to impress the group, or by adoption of 
a judgmental manner (God-like pose), or by showing insensitivity to the needs 
of participants and to what they are saying, or by driving the group too hard. 
These manifestations of 'power tripping' create distance between the facilitator 
and the group. In attempting to avoid the negative expression of the power 
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inherent in the role, facilitators adopt a humanist, caring approach, take cure 
to negotiate the program and any changes with the group, and create the 
conditions that allow the groups energies to be released and the guidance to 
allow then to work effectively on the task. The egalitarian value adopted by 
facilitators is revealed in the rules of treating all participants as equals, 
valuing contributions equally, transmitting empathy, and making decisions by 
consensus. 

A leadership focus 

Facilitators appear to adopt the conception of a leader as one who sets 
clear directions, is flexibly responsive to group and individual needs, and i£ 
prepared to step aside when the workshop momentum is progressing well. [ 

t 

The ability to motivate groups 

r 
t 

Facilitation requires an ability to generate, unlock, or inspire energy and 
interest within the group. The means for achieving this are by careful design 
of the workshop and selection of participants, the adoption of the values and 
rules embodied in the first two themes mentioned above, together with the 
transmission of enthusiasm about the worth of the project, transmission of 
positive expectations about the outcome, the display of faith in the group (e.g. 
through valuing group contributions), and a preparedness to adapt the process to 
the needs and interests of the group (by being sensitive and perceptive of 
group and individual needs). 

Good facilitation requires excellent searching and communication skill 

Searching skills include the ability to tease out the issues beforehand, 
including the politics and 'hidden agendas', the implementation realities and 
problems which bear upon curriculum development, and the ability to plan the 
workshop accordingly. Communication skills include the ability to listerl 
carefully, particularly to the 'music behind the words', and to be clear and 
succinct. ( 

i 

These themes are taken up in Table 5.1 which shows the general stages anc| 
steps in the evolution of curriculum research workshops, and the values and 
social rules which appear appropriate to each stage. The skills listed below 
are important in striving for the ideal of a democratic leadership style. 
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TABU*! 5.1 



STAGES AND STEPS IN CURRICULUM RESEARCH WORKSHOP PLANNING AND 
IMPLEMENTATION AND FACILITATORS' VALUE ORIENTATIONS AND SOCIAL 
RULES (SUMMARY) 

Stages and steps Value orientations/action guides 



Stage 1: Planning 

a) Research the issues 



b) Get aims clear 



c) Select or design 
workshop 



d) Choose participants 
carefully 



Know the background. Extract the 
the issues. Know the politics 
(the 'hidden agendas') affecting 
the project bnef or likely to 
generate tension within the 
workshop. 

Understand the issues and problems 
which will affect implementation 
of the outcomes. 

Know what the sponsor wants. 
Know what you are going to do in 
the workshop. Be able to 
communicate this clearly. 

(See Table 1.1 for questions 
which guide selection of 
various methods.) 

Know the method thoroughly. 
Be able to explain the method 
clearly and succinctly, 
including its 'ground rules'. 
Be prepared to be flexible 
in implementation. Know where 
you can bend the rules. Have 
contingency options. 

Get the 'right' people for the job, 
i.e. those in a position to 
know the area being studied. 
Choose participants with a 
good cross-section of 
k now ledge/sk i I Is/awareness. 
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Stages and steps 



Value orientations/action guides 



e) Check out, set up 
venue 

THE WORKSHOP 



Aim for comfort, ease of 
interaction. AIIcw for 
socialising. 



Stage 2: Orienting participants to the task and how it will be 
achieved (the method or process) 



a) Welcome participants 



Be friendly. Put people at ease. 



Begin building 'team spirit 1 and 
group cohesion. 



b) Explain aims/task/ 
process/ou tco mes 



c) Check out participants 1 
expectations 



d) Participants acquire 
skills and knowledge 
about the process 



Brief and orient participants 
to the planned process. Be 
clear and brief in explanations. 

Help people to 'see 1 where 
they are going. 

Listen carefully (throughout the 
workshop). Show that you are 
listening. Be open. State your 
expectations. Indicate what is 
able to be changed and what is 
not. Respond sensitively 
and perceptively to group and 
individual needs. Defuse tensions 
early. Set up a supportive 
atmosphere. Value all 
contributions equally. Do not 
be authoritarian. Avoid 
'power-tripping 1 . Attend to any 
problems which participants 
have in regard to the process 
being used. Be prepared to shift 
tack (be flexibly responsive to 
group and individual needs). Use 
consensus to make decisions. 

Participants need to understand 
the process in which they are 
participating. Teach the required 
skills to the participants, 
if necessary. 
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Stages and steps 



Value orientations/action guides 



Stage 3: Build momentum 



Attend to any problems which 
participants have in relation to 
the process being used. 
Let the group set the pace. 
Adjust the rules to the group. 
Make decisions by consensus. Do 
not value the task over people's 
needs. Maintain group cohesion. 
Lead by enthusiasm. Create a 
feeling of energy. Encourage 
participation. Be skilled in 
getting information from the 
group. Don't be intrusive 
but draw out silent members. 
Handle confrontations when they 
arise, sensitively and 
tactfully. Avoid emotional 
roadblocks to communication. 



Stage 4: The workshop in 'full-swing 9 



a) Plenary (whole group) 
sessions. 



Be prepared to step aside if 
the group is working well, i.e. 
has 'matured' to the task. 
Be prepared to change direction 
if th group becomes 
bogged-down. Assert control 
where appropriate. 

Attend to 'people needs'. 
Watch participants 1 comfort 
needs, lonitor time use. 
Schedule breaks when 
appropriate. Maintain group 
spirit anr' cohesion. 



b) Small group sessions 



In t\ task work, the 

tor may be more 
demanding (not satisfied with 
partial solutions). Monitor 
time usage. 

After the group leaders are 
selected (preferably by the 
group), ensure that they and 
the group understand the task. 
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Stages and steps 



Value orientations/action guides 



Stage 5: Concluding the workshop 

a) Summarise Summarise the workshop 

findings. 

b) Achieve closure Leave people with a sense of 

accomplishment. 

c) Get commitment Get commitment to outcomes 

(especially important with 

the Search Conference method). 

Stage 6: Follow-up 

Document and distribute reports 
of the workshop to participants. 



5.3 DESCRIPTION OF THE GENERAL STAGES IN CURRICULUM RESEARCH 
WORKSHOPS 

We will now trace the model through its various stages, amplifying a number 
of key points. 

Stage 1: Planning 

Research the issues 

It is vital in the planning stage to research the issues. It is important to 
know the background to the relevant issues and to have a general idea of the 
different backgrounds, interests, motivations (and factions) among the 
participants and therefore an idea of what information may come from the 
group and what tensions might arise. Clarify the aims of those commissioning 
the project and the project objectives. Talk to experts before the seminar. 
Try to find out what are the central issues at the structural level of the 
occupation, e.g. legal requirements, trade union concerns such as demarcation 
issues, and the issues at the student level. Is mobility within the occupatioji 
important? This requires the ability to liaise effectively with a variety ojF 
groups. Check the previous history of the project. Without this knowledge the 
appropriate workshop process cannot be selected. 

By knowing the issues, a good facilitator can lead a group into exploring 




these issues or can pick up important issues which arc below the surface or 
are hinted at by subtle cues. Moreover, the facilitator is in a better position 
to bend the rules to address key issues and to bring to the surface such 'hidden 
agendas' which might, if opened up, clear away obstacles for the 
implementation of the workshop outcomes or modify the program to 
accommodate them. The concept of 'hidden agendas' refers to those plans and 
motives that remain hidden but which can exert a powerful effect especially on 
the assessment of recommendations. Doing your 'homework' is vital for 
successfully eliciting relevant information from the group. 

Get the aims clear 

Knowing the issues enables the facilitator to get the aims of the workshop 
clear and to communicate these to the group. Groups can wander off the 
track and might require reminders about the aims. Participants also need to 
agree on the aims. Not only does the facilitator need to be a good persuader 
of others but also to be flexible in adapting the workshop process to the aims 
of the program and the needs of the group, first, by choosing participants 
wisely and second, by accommodating the group's wishes. 

Select or design the workshop 

Some key questions which guide selection of workshop types have been listed 
in Section 1.3. The essential decisions appear to rest primarily on whether a 
sufficient understanding of the nature of the industry or activity for which a 
curriculum is to be developed has been gained through preliminary investigation 
and/or exploratory research. Assuming that this has been done or the 
necessary knowledge is possessed, then the selection of group process methods 
might take account of the following points: 

. The question of whether the occupation is relatively 

stable or is undergoing change. If it is undergoing change, 
consider Search, Delphi or NGT group process methods. 
If it is relatively stable, consider DACUM. If the 
situation is complicated, consider a blend of methods 
(see Section 4.3). 

. Questions of whether the implementation difficulties are 
paramount. If so, consider Search or NGT group process 
methods. 

. The question of documentation needed for accreditation. 




Since group process methods rely on a relatively small 
sample, it may be necessary, especially if large sums of 
money are involved for curriculum development and 
implementation, to use a multi-method approach (see 
Chapter 4) and to follow this with a questionnaire for 
validation. 

A central concern for planning a workshop program, is to fit the program to 
the aims of the curriculum research project. Have a clear structure or 
structures planned, but feel free to modify them in the light of the group's 
needs or to vary the activities to allow change of pace. Include 'report-back 1 
segments where members of the group provide feedback to others (so that the 
pool of information generated is a 1 ailable to the whole group) and give 
opportunity for discussion and interaction. Include activities which cater for 
group maintenance (i.e. allow the group to work as an enthusiastic and united 
group and allow participants to contribute to the process where particular tasks 
can be successfully completed). Note that, in general, flexibility in design and 
implementation of the workshop is a central rule in curriculum research 
workshops. 

For Search Conferences, design the agenda carefully. Although the 
agenda/process for each Search Conference has the same format, they should 
be tailor-made to a particular group. Time constraints may apply and may 
require compressing two or more steps. Give an indication of timing on each 
stage as many participants are time-oriented. Plan Search Conferences to 
allow time at the beginning where there can be careful attention made to the 
expectations of the participants. For a OACUM workshop have the workshop 
follow a good rhythm, of around four main tasks per day, not lots of bits and 
pieces. 

Choose participants carefully 

Draw together a group of persons who are respected for their individual 
knowledge and ability, including being leaders in their field and good workers. ] 
Find out what each participant can offer in terms of specialist knowledge and [ 
interests. Make sure participants are telephoned beforehand and that the 
invitatior is confirmed by letter. ; 

Check out, set up venue 

Make sure (ho venue is com fore able, includes appropriate workshop space and 



ERIC 




a separate area for socialising, and arrange for the necessary facilities, such as 
refreshments and clerical/typing support services, and resources. Visit the 
venue before you finalise your choice. Arrange the room to maximise group 
sharing and group unity, i.e. put chairs in a semicircle, rather than in rows 
facing the front. 

Stage 2: Orienting participants in the workshop 

Welcome participants 

Start in a low-key manner, introduce yourself and others briefly, explain 
your role, thank people for coming. Explain the importance of the workshop. 

Explain aims/purpose/roles/tasks 

Explain the purpose and goals of the workshop, preferably in writing and 
repeat this at the start of the workshop. Be clear on what you are trying to 
do. Set realistic goals for yourself and the group. Where appropriate, provide 
background information to the participants before the workshop. If you think 
it is necessary, explain any alternative action which might be used to achieve 
the workshop goals, In particular, rtate what is open to be changed and what is 
not, but be prepared to negotiate with the group. Develop contracts with 
participants in the workshop in relation to their responsibilities, and clearly 
outline the outcomes expected from the workshop. 

Check out participants' expectations 

Attend closely at the beginning to the group's needs, especially the need to 
get to know each other and to resolve any uncertainties about the process in 
which they will be engaged. Clarify participants expectations (and state 
yours). Listen carefully to the exact words or nuances people use. 

Participants acquire skills and knowledge about the process 

It can be useful to provide an introductory training session at the start of 
the workshop to simulate key aspects of the task which participants will be 
asked to perform in order to ensure that all participants have the necessary 
skills and understanding and to generate a 'group 1 feeling. The facilitator may 
need to teach the skills which are required by the group. 
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During the workshop, when specific problems emerge which need to he 

solved, the facilitator must make the problem-solving task very clear and, 

preferably, display the task in writing. Here the facilitator's role is to help 
people to 'see 1 where they are going. 

Stage 3: Building momentum 

Provide motivation and support 

The aim of the facilitator is to successfully sustain the interest and 
involvement of the group. This can be achieved in part by establishing a 
positive climate, using sensitivity and tact, and leading by generating 
enthusiasm. Encourage a sense of pride in, and ownership of, the product of 
the group interaction. 

Create a supportive atmosphere in which individual and group inputs are 
valued. The facilitator who is gentle and caring with people, can put a group 
at ease. Show sensitivity to individual and group needs, and avoid alienating 
people. 

It is important not to force opinions on others and to keep task and people 
needs balanced. Do not let the task predominate over people or people 
overtake the task. Provide regular encouragement and positive feedback to the 
group. Demonstrate confidence but do not be dominant. Transmit positive, 
open expectations but try not to colour the outcome with one's own views, 
i.e. guard against transmitting one's expectations as to what the outcome will 
be. Instead, transmit enthusiasm that there will be a successful outcome. 
When the facilitator is not a task expert, he/she should display confidence 
about the process, but be tentative about the task content (listening carefully 
to the views of the expert participants). 

Work toward group cohesion 

Promote cohesion and keep the group united but active. A way of working 
toward group responsibility and the achievement of creative solutions is to 
encourage self direction and responsibility by the group. Draw out silent 
members but do not put anyone 'on the spot'. Encourage the group to identify 
solutions which suit them. Accept procedures determined by the group and, in 
particular, adopt procedures which meet the emerging needs of the group. 
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Momentum can be stifled if group emergent needs are not responded to 
sensitively and perceptively. Be prepared to change direction if the group 
becomes bogged down, and to liaise with the group to achieve the goals or 
tasks. Allow the group to grow and to set its own pace. Watch to see if 
anyone is expressing dissent non-verbally with what is being said. Be prepared 
to step aside if the group dynamic is flowing and relevant. Keep the 
momentum going. Do not be intrusive or tie the discussion down, or let 
anyone else tie it iown. When using small group sessions within the workshop, 
it is helpful in promoting group cohesion, allaying anxiety and pooling 
knowledge, to get the groups to report to each other at various times. 
Encourage self-direction and responsibility. 

Ensure participation 

A key to good facilitation is to ensure that all participants contribute to the 
workshop. Elicit responses from silent members. Make comments that are 
supportive and encouraging. Try to acknowledge all input as contributing to 
the total effort. Value all opinions equally - forget status and position. 
Watch carefully to see that people are not being cut off by other group 
members or are being excluded from contributing. Draw people out. Make it 
easy for them to contribute. Accept all comments initially (regardless of 
quality) so that a fair hearing of suggestions occurs. Allow equal time for 
comment. While trying to involve everyone be careful not to put anyone on 
the spot. Try to ensure equal participation in an unobtrusive fashion. Work 
toward stimulating effective group interaction and participation. 

Maintain focus 

Keep the focus clear by relating all new tasks to previous ones answering 
'what for' questions as they arise. Be task oriented and persistent in gaining 
tusk information. This requires having a good concept of what is the task. 
But vnlue peoples feelings over the achievement of the task. 

Use consensus 

Decisions of vital importance to the group and participants should be 
reached by consensus. Whenever you change the task or the pace, allow 
opportunity for clarification and or dissention. Face and attempt to resolve 
conflict. If conflict is serious and cannot be resolved within the workshop, 



schedule a break and take the protagonists aside to see if the conflict can be 
resolved in private discussion. Then, when the session resumes, state what 
agreement if any was reached and see if this is acceptable to the group. If 
the workshop has to be abandoned, at least the major obstacles to course 
development will be known. If serious conflict is anticipated, this should be 
examined before the workshop is conducted. 

Stage 4: The workshop in full-swing 

A. Plenary (whole group) sessions 

Step aside if the group is working well. Be prepared to shift direction if 
the group becomes restless or bogged down. Keep the task down to 
manageable proportions. Ask questions for clarification. If unsure, summarise 
what you understand about what has been said and seek confirmation as to its 
accuracy. If you spot an anomaly, hand it back to the group fjor 
clarification. Do the same if anyone asks a question which the group should 
answer. When writing down other people's ideas do not paraphrase unless you 
have checked most carefully with the originator of the idea on the accuracy of 
your version. 

Get feedback 

Provide opportunities for and encourage feedback from participants about 
everything. Allow diversions to the program only by consensus. Avoid long 
report-backs. Monitor use of time. 

Share the leadership role where appropriate 

Give opportunity for other participants to take the leadership role where 

appropriate. Do not allow anyone who is less skilled than yourself in 

facilitating the task to take over from you unless it is planned. 

Allow time for refreshment breaks and socialising 

Have frequent breaks, especially at critical periods in the workshop (when 
participants become tired or look like becoming bogged down or too tensej). 
Have light lunches and allow time for socialising. ! 
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Choose group leaders according to the needs of the seminar. Get group 
leaders 10 meet frequently during the session. Familiarise group leaders and 
seminar coordinators with the area under consideration, the procedures for 
running the seminar, the broad areas in which information might be obtained, 
and the timetable. Spend time to ensure that the group understands the task. 

Sea— le Coocludtm the worfcahon 
Summarise at key points 

Summarise the process at key transition points. Set up process checks (I.e. 
check participants perceptions of how they are feeling about the process). Get 
clear about what the participants expect and want. Establish if you have 
commitment to proceed. 

Achieve closure 

Closure must be ensured. Closure Is achieved, for example, by summarising 
Important points so that people feel that they have something to take away. 
Get closure on a workshop by reviewing the aims or expectations and 
summarising the workshop achievement. Thank the participants and praise 
where praise Is due. 

Value achievement 

Leave people with a sense of accomplishment. Make sure some concrete 
action Is decided. 

Follow up output 

Document and distribute all agreements, meetings, etc Follow up the 
output of group with further research as needed. 

Skills required by facilitators 

This section summarises the skills required by the facilitator. 
Communlcatloa skills 

The facilitator needs a high level of skill in communicating effectively. 
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This requires, among other things, the ability to have an open mind and to 
listen carefully and actively. One way of doing this is to show that you are 
trying to follow what people are saying and that you value what they are 
saylngt Maintain eye contact with the person who is speaking. Be sensitive to 
the verbal and non-verbal cues from the group. 

Make the relevance of activities clear (in relation to the overall task). 
Avoid Jargon and be precise. Have an open mind. 

Be tolerant, respectful and appreciative 

Acknowledge the views dxprfessed by participants even though they may not 
be personally acceptable. Be democratic. Value people for themselves. Avoid 
put-downs. Indicate that you respect participant's expertise. Be prepared to 
invest time In dealing with emotive issues so that they do not waste ever* 
more time later. Allow anyone to intervene at any time and acknowledger 
their concern, especially when you do not immediately act on it. Indicate yourf 
willingness to be interrupted. 

Be casual, relaxed, open and sincere 

Be casual, relaxed, low-key and concerned to put group members at ease. 
Be open to others. Answer questions frankly. Say what you are doing and why 
you are doing it. Be seen as fair minded, i.e. to seek other views without 
passing judgment. Keep formality to a minimum. Be honest and sincere in 
interactions with group members (do not work behind a facade). 

Transmit empathy 

Try to see things from the other's perspective: to stand in the other's 

shoes. Try to blend in with the group (including style of dress). Do not be 
seen as aloof. 

Be friendly and supportive 

Be friendly, supportive, encouraging and warmly responsive to people. Be| 
attentive. t 

Adopt a quiet, confident and non-defensive manner 

Be quiet (i.e. not dominant or authoritarian), and project a confident manner 
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and a sense of direction. 



Be fair Show appreciation of contributions. Be seen as fair minded. 

Be flexible: Adjust the process and the rules to the group and be alert to 
individual, group and organisational needs 

Adjust the workshop process to the aims of the meeting and to the needs 
and pace of the group. Pick up important issues raised by the group. Allow 
regular time-out from the task work to examine the process (by which the task 
is being achieved) and correct any problems. Value the group's efforts in this 
regard. Since there may be conflict about the process itself, allow questioning 
of the process. Bend the rules or change the direction or style of operation 
according to the situation. 

Be aware of the importance of the groups feelings and pick up group issues 
and suggestions 

Feelings are an important source of information. Be sensitive to the mood 
or feeling within the group. Have the ability to handle feelings and 
confrontation well (e.g. by defusing potentially tense confrontations between 
participants which is important in overcoming emotional road-blocks to 
communication). Conflict can be an important indicator of an underlying 
problem, which may relate to how a task should be approached and, if 
addressed tactfully, can allow the group to become productive. 

Work toward shared understanding 

Work toward a clear, shared understanding with the group. 
Show faith in the group 

Trust people to take responsibility for their own learning. Communicate, 
verbally and non-verbal ly, very positive messages about the outcome. Try to 
inspire interest and create a feeling of energy. 

Use control when appropriate 

Deal with dissent or resistance immediately it surfaces - listen especially 
carefully as there may be a misunderstanding which you can clear up or you 
may be doing something which needs to be put right immediately. Notice 
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when tensions are starting to arise and defuse them, e.g. using humour to 
lighten the situation or by introducing a break in proceedings. Just 
acknowledging the tensions may sometimes be sufficient. 

Counter productive aspects of facilitation 

Several destructive aspects of facilitation are summarised below. 
Power-tripping 

Avoid being overbearing, dominating, too directive or autocratic. Do not sit 
in judgement on the views expressed, or indicate that alternative ways of going 
about the task are a poor choice or override individual participant's needs. Djb 
not try to impress the group with knowledge. Do not push the group to your 
solutions or push the group too hard. | 

i 

Failure to provide clear direction \ 

i 

Do not give confusing instructions, set unrealistic goals, allow aimless 
discussion or let the process continue unmonitored. 

Failure to motivate groups 

Do not allow participants to become restless or dissatisfied, allow reporting 
to go on and on, make comments that are negative or critical, engender group 
hostility or resistance to learning, or break the flow of the group. 

Lack of sensitivity, insight, care and consideration of people and their 
feelings 

Do not over-emphasise the task over peoples needs. Don't leave people 
wondering what is wrong with themselves. Do not cut people down. 

Conclusion ^ 

r 

The foregoing shows that considerable skill is needed to effectively facilitate 

or lead a curriculum research workshop. This is especially the case when 

t 

conflict situations arise, hence planning is important. I 

i 

There is a need to learn more about how conflict can be handled not bjr 
choking it off or sweeping it away but recognising it, working with it, and 
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trying to find solutions which relate to the goal of achieving quality and 
relevance in the curriculum. The facilitator must be able to recognise 
irreconcileable conflict for what it is and abandon or postphone further work 
until the conflict is resolved. 
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CHAPTER 6 



SOME METHODOLOGICAL ISSUES IN THE 

USE OF GROUP PROCESS METHODS 

6.1 ISSUES ASSOCIATED WITH RESEARCH METHODOLOGY 

In Section 1.3 these issues were listed as (A) the relative strengths of 
qualitative versus quantitative methods, (B) the problem of sampling bias, (C) 
the involvement of educators and practitioners in the research process and (D) 
the question of who controls the workshop output? j 

t 

A. Strengths of qualitative versus quantitative methods I 

Some problems with the use of quantitative methods were discussed itv 
Section 2.2, in particular (1) the problems of selecting a representative sample 
when the industry is large or not clearly defined, (2) the question of how 
useful is the data when the response rate is poor, a problem which does not 
become known until after a considerable investment in time has been made, 
and (3) the problems of translating a fragmented view of work done in the 
industry (e.g. data on tasks done by how many) into an instructional design. 
This, it was noted, is usually a subjective process liable to bias. 

Qualitative methods, on the other hand, usually suffer from lack of cases to 

support the conclusions reached. Offsetting this is the strength of qualitative 

methods, when properly used, to do a better job of describing the issues to be 

taken into account in course design. However, this too can be a matter for 

debate. There are strengths and weakness for both qualitative and quantitative 

methods. Qualitative methods and group process methods in particular appear 

to be faster and particularly suitable for course revision where the extent of 

demand for places is known. In starring up a course in an entirely new area£ 

the size of the enrolment demand probably should be assessed usin£ 

I 

quantitative methods, particularly if arguments must be presented about the 
costs of implementing the course. !j 

Qualitative methods work well when considerable expertise already exists 
among the educators and the main questions with which they are concerned are 
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the larger issues of 



i) Thrust (i.e. course aims). 

ii) Scope (what should be in or out?) 

iii) Weights of emphasis (where should these lie?) 

iv) Sequencing (what should come before what?) 

When the larger questions are known, it becomes easier for the 
educationalists to pinpoint their knowledge gaps and to move to plug these, 
i.e. through industry visits, study of equipment manuals, staff development and 
so on. 

As noted in Section 2.2 qualitative methods can be used to account for the 
patterns resulting from quantitative analysis and so to round out the picture 
given by quantitative analysis. For example, qualitative analysis can explain 
why a group of skills is important whereas frequency analysis merely says how 
many people are performing the skills. Questionnaire methods as used in 
curriculum research largely reflect the limitations of the knowledge of the 
designer. Qualitative methods are informed less by a notion of expert 
knowledge already held and more by a questioning approach: 'What am I 
missing that I need to know?', 'Where must I look next?' then, 'What 
quantification do I need to show how important are the various spheres of work 
in the industry under study, from the viewpoint of designing the curriculum? 
This orientation makes qualitative methods particularly suitab' as a prelude to 
questionnaire design where the main purpose of the questionnaire is to count 
occurrences of skills that the more free ranging qualitiative methods have 
shown are in demand so that, in the first instance, the data can be related to 
the broad instructional design questions listed in items i) to iv) above, and in 
the second instance, to questions of emphasis to be placed on learning skills 
versus learning underlying concepts. 

It has also been noted (Anderson, 1986) that when teacher group sessions are 
held on how to restructure a course, it is much easier for them to wcrk on 
data of the kind enumerated in i) to iv) above, than to wade through a 
statistical print-out of frequencies of tasks done by how many. When speed is 
of the essence, this is of considerable advantage. Fine tuning can then take 
place when the course is implemented as it is only when a syllabus is tested in 
the classroom that some of the problems to do with the transmission of skills 
and underlying concepts can be resolved. 



B. The problem of sampling bias 
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Qualitative methods, when they rely on only a few cases such as when 
'typical applications' sampling is used or on it small number of participants 
when group process methods are used, are particularly subject to sampling 
bias. The problem of sampling bias can be viewed from a number of 
perspectives. 

One way of viewing how bias can be introduced is to use a social class 
perspective. Given that group process methods enjoin a group of people with a 
professional-managerial class background in designing curricula for people with 
essentially a working class background, one might ask: "What are the social 
class interests of the former that may become woven into the curricula for the 
latter which may not reflect their (working class) interests? The DACUM 
process, looked at from this perspective, may do a good job of developing 
curricula to teach working class people to be good operatives but not good 
managers because management is not seen as the preserve of the working 
class. In times of rapid technological change one might ask: 'How useful is 
DACUM in teaching skills related to how to manage change? Or do any 
curriculum research methods allow workers to have a say in the educational 
implications of the introduction of new technologies with their attendant 
problems of 'de-ski lling 1 ? In this connection, among the group process methods, 
the Search Conference method at least mixes employers, educators, 
practitioners and students in the curriculum research process. 

Another form of bias is that which can result from the psychological 
dynamics at work within group process methods. Perhaps the most serious of 
these psychological dynamics concern the getting of consensus. The pressure 
on participants to conform to the consensual in a group setting can be quite 
strong, especially when a view is strongly put by people of high status such as 
a major employer. The advocates of the Nominal Group Technique claim that 
their method is expressly designed to avoid such influences. Researchers who 
have studied the Delphi process, particularly Weaver (1971) note how the push 
toward achievement of consensus can influence responses of the Delphi panel. 
In the Search Conference approach, the role of the facilitator is especially 
delicate as it is easy for the facilitator to be especially powerful in 
sidetracking conflict issues in the belief that attaining consensus is more 
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important. 



C. The involveme nt of educators and practitioners in the research 

In our view it is essential to involve educators in the research process. 
Where this has not been done, e.g by use of a DACUM process whicl. excludes 
teachers, there has invariably arisen strong resistance to the curriculum which 
teachers are then asked to implement. The Search Conference method includes 
educators and practitioners. With the Nominal Group Technique it has been 
suggested by Hegarty (1977) that teachers should review the results of the NGT 
workshop before implementation of the results. This is also advocated for the 
DACUM method (Lyons, 1984). 

D. The question of who controls the workshop output 

This is very much concerned with nuestions of who owns the curriculum: the 
institution which offers it or the client? The Search Conference method, as 
previously noted, joins educators, practitioners and students ir, the research 
process and has an added advantage, the action groups, for following through 
on issues raised and solutions generated. The preferred solution to this issue, 
in our view, is to involve both teachers and students in the research process 
and, if that is not possible, to ensure that the teachers will have an 
opportunity to review the workshop output. This also makes good sense from a 
research point of view: the teachers may spot something missing. 

6.2 CONCLUSIONS AND FUTURE DIRECTIONS 

Some conclusions from the study are: 

(a) the importance of completing sound preliminary 

investigation; 

(b) the need to use multi-method approaches which address 
questions of occupational structure and change at the 
broad level, in relation to course aims, scope, depth 
and course structure, and questions of specific 
competencies needed in order to produce quality 
curricula without obsolescence; 

(c) the value to be obtained in using the methods described 

in combinations which exploit their strengths and, 
therefore, strengthen their weaknesses; 



(d) the importance of enhancing the skills of curriculum 
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developers in the use of group-process methods; 



(e) the need to obtain comprehensive case study data on 
applications of the methods described in order to assess 
cost, time, expertise needed, and the quality of data 
produced. 

Where do we go from here? Speaking about the pursuit of an 'ideal method 1 
for job analysis, Levine et al. (1981:51) note that one limitation of current job 
analysis technology is its relative inability to deal with the job as it should be, 
rather than how it is now, and with the job's changes over time. 

Throughout this study we have drawn attention to the importance of using 
two levels of data analysis. One level addresses occupational structure and 
change and the other addresses individual educational needs, keeping in mind 
the student's need to learn how to cope with the future changes in technology 
and career. I 

i 

There is probably no v 'ideal' method. However, a promising question t<Ji 
pursue would ask what is the ideal blend of methods? To answer this question 
we need empirical results from imaginative blends of methods. There is a 
need for detailed documentation of the methods, especially exploratory 
(qualitative) approaches, the Search Conference, Delphi and the Nominal Group 
Technique, and preferably in comparative studies between States. We need to 
know more about the strength of these methods in TAFE applications. We also 
need to know what wording of questions works best with Delphi and NGT, and 
what research questions work best with exploratory methods. 

A number of promising combinations of research methods have been 
identified such as joining Search and DACUM, Delphi leading into DACUM and 
possibly the Search Conference Method. Linking Delphi and Search, while 
controversial with Search advocates (in the Search Conference rationale the 
participants are the 'experts' rather than outside 'experts'), would overcome a 
weakness of Search. This weakness is that the participants, although they may 
be in the key vantage position to deal 'expertly' with implementation realitiesj 
may not be so expert on 'futures' impacting on the present. Industry forecast 
data derived via Delphi could be given to Search Conference participants 
beforehand as briefing notes for reference. However, the introduction oj 
'expert' commentary into the search process is controversial according to some 
Search Conference devotes. Nevertheless the search process appears to be 



ERLC 



151 

ISO 



weakened if poor data on trends emerge in the 'future scan' phase. 



On shorter projects, a promising method is to spend time, following the 
preliminary investigation, on exploratory (qualitative) research addressing 
occupational structure and change with a view to defining the main thrust of 
the course (aims, content, sequencing, course structure), and then either move 
directly onto curriculum design where fine detail questions are left to the 
syllabus writers to answer as needed, or follow with a questionnaire. This 
approach, with the addition of a questionnaire to provide back-up validation 
and expansion of the exploratory research, has been tried successfully and is 
suitable for small amounts of teacher release, e.g. 8 hours per week spread 
over a period of two or three terms. 

The conduct of curriculum research group methods is a complex skill. While 
most workshops run smoothly, difficulties have been reported with problems of 
conflict within the group due to factional split. We suspect, although this has 
not been reported in the literature, that difficulties can arise in workshops 
with the definition of the aims of the process. Conflict may not be voiced 
but may be expressed by passivity - an unwillingness to contribute. Since 
sample sizes are necessarily small, one or two people not responding can skew 
the results. Note especially the effort which skilled group leaders put into 
encouraging full workshop participation. It is not recommended that curriculum 
developers unfamiliar with group process work leap in head-first. It is 
recommended that they either do a trial run, sit in on a group process method, 
or at least have a co-facilitator (partner) to help with the recording. 
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APPENDIX A 



DESCRIPTIONS OF SOME QUESTIONNAIRE METHODS 
Introduction 

This appendix describes several types of questionnaires developed for job 
analysis. 

The Position Analysis Questionnaire (PAO) 

The PAO is a structured job analysis questionnaire consisting of 18f| 
elements of a 'worker oriented 9 nature. It aims to identify and measure th& 
underlying structure of the various components of work (Smith and Hakelj> 
1972). Youngman et al. (1978) describe the PAO method as a factor analytic 
approach giving the following five broad dimensions of work: having 
decision-making, communication and social responsibilities; performing skilled 
tasks; being physically active/related environmental conditions; operating 
vehicles/equipment; and processing information. 

The Job Components Inventory (JCI) 

Banks et al. (1983:57) have developed a new job analysis technique which 
takes about one hour to administer. The technique involves examining the use 
of tools (covering 220 tools and pieces of equipment); the physical and 
perceptual requirements (23 items including strength, co-ordination, dexterity, 
reaction time and selective attention); mathematical requirements (127 aspects 
of mathematics including elementary algebra and trigonometry with practical 
applications such as work with plans and drawings); communication requirements 
(22 items dealing with preparation of reports and letters, use of coding 
systems, dealing with complaints, receiving written communications and othe* 
features of interpersonal relations), and decision-making and responsibility 
requirements of work (9 items covering decisions about methods of work, orde$ 
of work, standards and related issues). || 

With the aim of measuring agreement between job holders and supervisors, 
and assessing discrimination Detween occupational areas, job titles and 
organisations where the dependent variable was the total number of components 
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used by each job holder, section by section and in total, the method was 
tested on four job titles from engineering work (grinding machine 
setter/operator, drilling machine operator, milling machine operator, and 
semi-automatic lathe operator) and four job titles from clerical work (rating 
clerk, library assistant, mail room clerk and shorthand typist). Banks et a I. 
(1983) reported a high level of agreement between job holders and supervisors. 
As would be expected, they found significant differences between JCI profiles 
of office and engineering occupations and between job titles in the 
occupational areas. The method reflects the variety and range of requirements 
of these particular jobs. For example, clerical work was found to require a 
wider variety of tools and equipment and to utilise more mathematical and 
communications components than engineering, but the two work areas did not 
differ significantly in perceptual, physical and decision-making requirements. 
They conclude that 'the JCI generates occupational components common to a 
wide range of jobs which could form the basis of broad based training or for a 
"core" curriculum, depending on the jobs studied 1 ((1983:64). 

CODAP 

The CODAP program (Comprehensive Occupational Data Analysis Program) is 
a suite of computer programs for analysing occupational data. Goode (1981) 
describes the process as placing prime importance on data about individual 
work positions taken from the job holders themselves. The crux of each survey 
questionnaire is the job inventory: a list of all the tasks which coJd be 
expected to be performed by any of the personnel working in the occupation 
being surveyed. A task is a 'specific unit of work which can be described 
using a single verb'. The tasks in the inventory are grouped into duties, a 
'duty being a broad category of work involving a number of related tasks'. 
Each questionnaire makes provision for the respondent to score each task 
against a numerical scale. 

Background information is gathered which includes data about the individual 
and about the job in general. These variables include rank (the method was 
developed in the Armed Services), employment category, experience levels (such 
as time in rank, time in employment category and time in present job), courses 
completed, types of equipment worked on and tools used, and questions about 
job factors which are essentially morale indicators and include the demands the 
job makes on the incumbent. 
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The most important or 'primary 1 factor is time spent on each task. The 
CODAP program then uses these ratings to compute an estimate of the 
percentage of working time which the individual spends on each task. 
Secondary task-factor data are collected using a survey instrument based on 
the same time-spent data. The respondents rate each task on the relevant 
factor against a scale provided, and the arithmetic average of all raters 1 
ratings on each task is taken as the measure of the factor for that task. 

The respondents are normally required to rate all tasks in the inventory 
rather than only those which they personally perform. Other secondary 
task-factors include task difficulty - the time needed for an individual to leaiti 
to perform the task adequately; consequences of inadequate performance - a 
measure of the seriousness of the probable consequences of inadequate 
performance of a task; and the task delay - a measure of how much delay can 
be tolerated between the time the worker becomes aware the task is to be 
performed and the time he or she must commence it. 

In order to tap a broader knowledge of the entire occupation being surveyed, 
the first line supervisors are used as raters. The CODAP program also 
provides a statistic indicating the reliability of the measures obtained, including 
information which identifies a rater who may not have followed the instructions 
properly. The CODAP program can display the secondary task factor 
information beside the primary factors of 'percent performing' and 'percent 
time spent'. Goode (1981) sums up the CODAP program as being a 'powerful 
tool for providing information which can be used in determining training 
requirements'. 

The CODAP method benefits from the possibility which exists in the Arme<j 
Services of 'mustering' respondents and directing them to complete th| 
inventory with the aim of achieving a near 100 percent response ratej 
Moreover, it is not clear at this stage whether the CODAP program provides & 
means of data analysis which is not available on any of the main-frame 
computers, e.g. Statistical Package for the Social Sciences (SPSS), Genstat, or 
any of the Multivariate Analysis of Variance programs. 

Functional Job Analysis Technique (FJA) 

The FJA technique is described by McCormick (1979) as 'both a conceptual 
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system for defining the dimensions of worker activity and a method of 
measuring levels of worker activity 1 . He cites Fine and Wiley (1971), who 
described the characteristics of Functional Job Analysis, as follows. 



1. Functional distinctions must be made between what pets 
done and what workers do to get things done. 2. What workers 
do, insofar as their job context is concerned, they do in relation 
to three primitives: Data, People and Things. 3. In relation to 
each primitive, workers function in unique ways. Thus, in relation 
to things, workers draw upon physical resources; in relation to 
data, on mental resources; and in relation to people, on 
interpersonal resources. 4. All jobs require the worker to relate 
to each of these primitives in some degree. 5. Although the 
behaviour of workers on the tasks performed by them apparently 
can be described in an infinite number of ways, there are only a 
few definitive functions involved. Thus, in interacting with 
machines, workers function to feed, tend, operate, or set up; and 
in the case of vehicles or related machines, to drive-control 
them. Although each of these functions occurs over a range of 
difficulty and content, essentially each draws on a relatively 
narrow and specific range of similar kinds and degrees of worker 
characteristics and qualifications for effective performance. 6. 
The functions appropriate to each primitive are hierarchical and 
ordinal proceeding from the simple to the complex* Thus, to 
indicate a particular function, (say compiling data) as reflecting 
the requirements of lower functions such as comparing and 
excluding the requirements of higher functions such as analysing 
(McCormick, 1979). 

The FJA approach provides for the analysis of jobs in terms of two worker 
function scales, namely, the level and the orientation of involvement with the 
three hierarchies (data, people and things). The level of involvement is 
indicated by the specific function of each hierarchy that is considered to be 
applicable to the job in question. The level of involvement of the worker is 
expressed in terms of a percentage applied to each of the three functions 
Jdata, people and things). For example, for a task in the social welfare field 
which requires the worker to fill out a standard interview form and which 
entails asking questions, listening to responses, and writing answers where some 
leeway is allowed as to the sequencing of the questions, the functional job 
analysis for the task would break it up as follows: 



Area Functional level Orientation 

Data Copying 50% 

People Exchanging information 40% 

Things Handling 10% 
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Note that the heading 'orientation' refers to the proportional relationships of 
the function to the three areas (data, people and things). Other scales which 
have been used with the FJA method provide for characterising each job in 
terms of worker instructions, reasoning development, mathematical development, 
and language development. Another important point about the process is that 
in writing task statements the action verb is modified by the means (tools, 
methods, equipment) required by the immediate objective of the action (if 
different from the result expected from the action), and by some indication of 
prescription/discretion in the worker instructions (McCormick, 1979). Ry 
modifying the action verb in task description, the FJA method appears to offer 
an expansion of the treatment of action verbs used in the DACUM method. 
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APPENDIX B 



SEARCH CONFERENCE ON FUTURE DIRECTIONS 
FOR TRADE EDUCATION IN GRAPHIC REPRODUCTION 
B-l INTRODUCTION 

This review of the NSW TAFE Graphic Reproduction Trade Course aimed to 
set the broad directions for trade education in graphic reprodut tion, with 
particular focus on questions of course aims, course structure, rmin content 
areas and, the larger questions associated with subject sequencing. 

The data collection phase included two linked steps: (1) exploratory industry 
visits which included observation of a cross-section of applications o r 
contemporary Graphic Reproduction technology and (2) the conduct of a Search 
Conference on 'Future Directions For Trade Education in Graphic Reproduction.' 

Use of these methods, together with a series of teacher works! 3^s to 
translate the data into a revised curriculum, enabled a major reconstruction of 
this trade course to be implemented in one year from commencement of the 
review. 

The project was carried out by Mr Bob Snedden and Mr Craeme Murray, 
teachers of Graphic Reproduction. Curriculum consultancy was provided from 
Directorate of Studies, NSW TAFE (see Anderson, 1986). 

The steps in the implementation of the project were 

STEP I. Preliminary discussion with the Head of School. 

STEP 2. Detailed discussion with the teachers released for the project to 
review issues and formulate research plan, 

STEP 3. Discussion of the research plan with key decision makers of the 
school (the Consultative Team). 

STEP 4, Conduct of exploratory industry visits. 

STEP 5. Report of preliminary findings to the Consultative Team. Decision 
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ratified to use a Search Conference procedure as the next and main data 
collection method. 



STEP 6, Conduct of Search Conference. 

STEP 7. Report of results to Consultative Team. 

STEP & Formal involvement of the teaching staff in the curriculum 
development process via a series of teacher workshops and individual and group 
submissions. 

STEP 9. Review of teacher input and development of syllabus. 

The essence of the consultancy model outlined above was involvement of 
decision makers within the school in all planning decisions and in review of tHe 
results from each planning decision. From the outset it was agreed that (|) 
the major focus of data collection would be on the broader questions of course 
aims, scope, weights of emphasis and subject sequencing, and (2) that tfje 
research plan would be flexible; the decisions as to what main data collection 
procedure to be used would be made after the exploratory work had shown 
what was the best way to proceed. The involvement of the general teaching 
staff began after the Search Conference. The rationale for this was that it 
would be a more efficient use of the teachers 1 time if they were involved 
a f ter data was available on what industry viewed as the direction that the 
course should take. 

B-2 CONFERENCE PLAN 

The facilitator introduced the conference as 'an examination of future 
dire ns for trade education in graphic reproduction with the aim of deciding 
(1) "'hat should be the thrust of the course (i.e. the broad aims), (2) what 
should be the scope of the course, in terms of what should be in and what 
should be out, (3) what weights of emphasis should there be on the various 
content areas, and (4) if possible, what should come before what (i.e. 
sequencing) 1 . Throughout the conference, participants comments were recorded 
on large sheets of paper which were displayed around the walls of the room. [ 

I 

IT'S actual sequence of questions used in the Conference varied from the 
original agenda because the participants were reluctant to divide into small 
working groups. The program evolved as the conference proceeded and is 
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summarised below. 



DAY 1; Evening session 5.30 to 9.00 p.m. with meal break at 7.00 p.m. 

1. What will Graphic Reproduction be like in 5-10 years time? (FUTURE 
SCAN PHASE) 

2. Looking back over the 'future-scan 1 , what patterns are apparent? (List.) 
DAY 2: 9.00 a.m. to 400 p.m. 

3. What are the current strengths and weaknesses in either the trade course 
or in employers practices in relation to training? 

4. What knowledge is needed by trade students? (General scan of required 
knowledge. 

5. What are the characteristics of an 'ideal' graphic reproducer in view of 
future trends/changes? (See Note 1 below.) 

6. List the major areas in which knowledge is needed now and in the next 
five years time? 

7. For the course in Graphic Reproduction, what must be emphasised 
overall? (See Note 2). 

8. What computer skills are needed and where must the emphasis lie? 

9. Brief session on: "Where do we go from here', including how the data 
would be used, and concluding remarks. 

NOTES: (1) After a a general review of knowledge areas 
which the course should cover, a suggestion was 
made by one of the participants that the 
characteristics of the 'ideal' student should be 
listed. This was very effective in presenting an 
integrated picture of the type of graduate which 
industry participants believed would meet the needs 
of the emerging technology. 

(2) Most of the participants had a detailed 
knowledge of the current course and some had direct 
responsibility for supervising apprentices. 
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B-3. COMMENTS ON THE SEARCH CONFERENCE 



The use of an abbreviated Search Conference duration of ten hours rather 
than the more typical threes day format was influenced by the belief that 
industry representatives would be hard pressed to find three full days to 
spare. In truncating the conference, the objective was to strip the method 
down to its minimum essentia^ whilst attempting to retain those elements 
which encourage the involvement of the participants which is claimed to be a 
strength of tile method. Those elements are: (1) the inducement to 'lateral 
thinking 1 provided by commencing with a 'future scan 1 of trends and forces 
affecting the occupation and (2) the encouragement of equal participation and 
of the group contributing to the way In which they will work toward the goal. 
Participant involvement was indeed generated: the conference formed into such 
a cohesive group that efforts to induce them to divide into smaller working 
groups as intended, failed, necessitating a change in the conference agenda. 

Experience of the group dynamics in this particular conference allows some 
issues in the conduct of short Search Conferences in curriculum applications to 
be noted. The Search Conference facilitator needs to be nimble and able to 
adapt the process, on-the-run so to speak, according to the types of issues 
emerging and the subtleties in the group's interaction patterns. This is not for 
the unprepared. As noted in Section 2.2 the underlying philosophy of 
'searching' must be understood. This requires an understanding of the wSearch 
Conference philosophy which rests primarily on the notion that the participants, 
not the facilitator, will define the direction which the conference will take. It 
also requires an understanding of the principles of exploratory research. 
Moreover, a considerable demand is placed upon the facilitator of such 
conferences to adapt the conference format in ways that participants see are 
relevant and to obtain their consent on what work they will do next, whilst 
keeping the conference on track as far as curriculum development purposes are 
concerned. In this Search Conference it was necessary for the facilitator to 
rapidly scan the output from each segment while participants were taking a 
refreshment break and evolve a pertinent focus question for the next segment. 

When devising focus questions, avoid the temptation to anticipate what the 
group might be ready to do in two hours time. Until the group have agreed 
to work on a particular focus question, then probed it in the ensuing group 
session, you will not be in a position to know what the next focus question 
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could be. 



Some of the more difficult questions in plowing a Search Conference 
include how to choose the right people and how many to invite. Around thirty 
participants is considered ideal. The question of whom to invite is dependent 
on the research aims. To generate a good future scan' requires that 
participants be in a position within their organisations to relate changing 
technological and social patterns in the market place which affect the demand 
for the firm's gcods and services, to the expertise needed within the firm for 
it to remain competitive. The ideal Search Conference p -ticipant, therefore, 
is one who is closely attuned to the requirements for technical skills within the 
firm, is engaged in a management level capacity in monitoring changing 
patterns in marketplace demand, and is contributing to the continual adaptation 
of the firm to meet these changes. 

If participants are carefully selected, then the conditions for successful 
group interaction will be present since participants will be able to see how 
other people in a similar position are grappling with the complexities of the 
industry the 'external environment' (to use Search Conference phraseology). 
These complexities are an inherently interesting dilemma for managers and it is 
hard for them to resist being drawn into the type of discussion which takes 
place during the 'future scan' phase of a Search Conference. 

Planning for the number and type of participants should anticipate a 5 to 10 
percent drop-out due to last minute work pressures on industry 
representatives. It is especially important to be able to replace drop-outs at 
the last minute. The technique could be used of pairing participants according 
to the segment of the industry which it is desired to represent, as achieving a 
good cross-sectional representation from industry is crucial to the quality of 
data generated. Accommodation and catering must be flexible in case all who 
are invited turn up. 

It appears that in cases such as Graphic Reproduction where the industry 
grouping wa? relatively homogeneous, the method does have potential for 
drawing out the main trends and issues and enables this information to be 
available quickly - in a matter of days - for curriculum development purposes. 
The shortened form of Search Conference as used here does not leave 
sufficient time for producing fine detail on tasks and performance levels. 
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APPENDIX C 



OBSERVATIONAL STUDIES OF HIGHLY SKILLED 

OPERATORS IN KEY WOkK A P \ 

This method could provide the research information for curriculum 
development in an appropriate occupational area, preferably a small, knowledge 
intensive area, or could form a part of a combinatory approach. 

The method entails an intensive study of skill masters working in key phases 
of a process, whether in an industrial or a service area. It requires k 
'co-researcher 1 relationship to be established between the curriculum developer 
and the skill master. In this approach, both parties contribute to the method 
by which information will be obtained. The curriculum developer would supply 
concepts from the social science area including research procedures derived 
from occupational research methods such as the use of the Critical Incident 
Technique or application of theoretical insights into the nature of 
problem-solving expertise. 

The purpose of the study would be to explicate the theory knowledge (i.e. 
knowledge 'how 1 ) as well as the practical knowledge/skill (i.e. knowledge f that f ). 
The principles entailed in problem-solving for the particular work function 
would be drawn out, together with such other trial-and-error based knowledge 
which enables a broad view of the work function to be developed. The method 
would need at least two observation/interview sessions with each key operator 
and may be augmented by other methods such as the keeping of a diary by the 
job holder. 

The sampling strategy would be to select as many key operators as is 
needed to obtain a picture of the genera! work area. Use of a tape recorder 
and/or videotape recorder could become part of the developed course. ' 

When used in combination with other methods, this approach would be seeij 
as an ideal, exploratory tool. Alternatively, where a DACUM method is use<| 
first, it would identify the key functions in each main duty area. Validation of 
these key functions would be carried out by a study of 'skill masters'; a study 
which is very likely to add useful detail. 
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The method would require a high skill level in the curriculum developer. 
Where the curriculum developer is Joined by a teacher, this could constitute an 
excellent staff development project for the teacher. 
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APPENDIX D 



A DACUM CHART FOR JEWELLERY REPAIR WORK 

A completed chart for jewellery repair work is shown below. The nine 
duties or general areas of competence are set out on the right hand side of 
the chart. The skills, which arise out of the DACUM session analysis of the 
duties, are set out on the left of the duties. 

The left-to-right sequencing of the skills are an hdication of the order in f 
which the student should learn the skills. The vertical alignment of skills! 
would be used depending on the course style to be adopted - for example;;? 
whether modular or subject based. In any case the vertical alignment of the' 
chart is an indication of those skills that need to be learned at about the same 
time. 

The square box space next to each skill number (e.g. 02.13) can be used if 
desired by the course planners for the inclusion of either L, M or S. L means 
•limited 1 level of competence is required; M means 'moderate 1 level of 
competence is required; S means 'skilled 1 level of competence is required. 

(NOTE: To assemble the chart depicted on the following 
two pages, copy and assemble by joining the two pages 
according to the number coding 1,2,..., 10.) 
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